Methods and apparatus to correct errors in audience measurements for media accessed using over-the-top devices

ABSTRACT

An example system comprising at least one memory, programmable circuitry, and instructions to cause the programmable circuitry to obtain first time information corresponding to one or more first times that a panelist accessed first media via a television, obtain second time information corresponding to one or more second times that a household accessed second media via an over-the-top device, the second time information matching the first time information and associated with corrected demographic information of members of the household, determine that a first genre of the first media matches a second genre of the second media, determine that a member of the household accessed the second media via the over-the-top device based on historical media access events of the panelist, and attribute the access of the second media to the member of the household to create a corrected demographic impression record.

RELATED APPLICATIONS

This patent arises from a continuation of U.S. patent application Ser.No. 17/676,689, which was filed on Feb. 21, 2022, which is acontinuation of U.S. patent application Ser. No. 17/027,398, which wasfiled on Sep. 21, 2020, now U.S. Pat. No. 11,259,086, which is acontinuation of U.S. patent application Ser. No. 16/525,218, entitled“METHODS AND APPARATUS TO CORRECT ERRORS IN AUDIENCE MEASUREMENTS FORMEDIA ACCESSED USING OVER THE TOP DEVICES,” which was filed on Jul. 29,2019, now U.S. Pat. No. 10,785,537, which is a continuation of U.S.patent application Ser. No. 15/956,092, entitled “METHODS AND APPARATUSTO CORRECT ERRORS IN AUDIENCE MEASUREMENTS FOR MEDIA ACCESSED USING OVERTHE TOP DEVICES,” which was filed on Apr. 18, 2018, now U.S. Pat. No.10,368,130, which is a continuation of U.S. patent application Ser. No.14/967,355, entitled “METHODS AND APPARATUS TO CORRECT ERRORS INAUDIENCE MEASUREMENTS FOR MEDIA ACCESSED USING OVER-THE-TOP DEVICES,”which was filed on Dec. 13, 2015, now U.S. Pat. No. 10,045,082, andclaims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional PatentApplication Ser. No. 62/188,380, entitled “METHODS AND APPARATUS TOCORRECT ERRORS IN AUDIENCE MEASUREMENTS FOR MEDIA ACCESSED USING OVERTHE TOP DEVICES,” which was filed on Jul. 2, 2015. U.S. patentapplication Ser. No. 17/676,689, U.S. patent application Ser. No.17/027,398, U.S. patent application Ser. No. 16/525,218, U.S. patentapplication Ser. No. 15/956,092, U.S. patent application Ser. No.14/967,355 and U.S. Provisional Patent Application Ser. No. 62/188,380are hereby incorporated herein by reference in their entireties.Priority to U.S. patent application Ser. No. 17/676,689, U.S. patentapplication Ser. No. 17/027,398, U.S. patent application Ser. No.16/525,218, U.S. patent application Ser. No. 15/956,092, U.S. patentapplication Ser. No. 14/967,355, and U.S. Provisional Patent ApplicationSer. No. 62/188,380 is claimed.

FIELD OF THE DISCLOSURE

This disclosure relates generally to monitoring audiences of media and,more particularly, to methods and apparatus to correct errors inaudience measurements for media accessed using over the top devices.

BACKGROUND

In recent years, over-the-top devices have become a primary source formedia presentation. Media delivered through over-the-top devices isaccessed from Internet sources via Internet protocol (IP)communications. Over-the-top devices may be used by any users orhouseholds having Internet access. For example, users can useover-the-top devices to access live programming, video on demandservices, user-generated media, and/or other types of media.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example apparatus to correct demographicimpressions, assign viewership, and predict demographics frominformation obtained from over-the-top (OTT) devices and databaseproprietors.

FIG. 2 illustrates an example OTT registration phase to generate anidentifier for an OTT device and associate the identifier withdemographics of a user that accesses media via the example OTT device.

FIG. 3 illustrates an example impression collection phase to collectdemographic impressions corresponding to the example OTT device of FIGS.1-2 .

FIG. 4 illustrates an example demographic corrector of FIG. 1 to correctcollected demographic impressions.

FIG. 5 illustrates an example viewership assigner of FIG. 1 to assignviewership to corrected demographic impressions.

FIG. 6 illustrates an example demographic predictor of FIG. 1 to predictdemographics for impressions not corrected by the example demographiccorrector.

FIG. 7 is a flow diagram representative of example machine readableinstructions that may be executed to implement the example demographiccorrector of FIGS. 1 and 4 .

FIG. 8 depicts another flow diagram representative of example machinereadable instructions that may be executed to implement the exampledemographic corrector of FIGS. 1 and 4 .

FIGS. 9A-9C depict another flow diagram representative of examplemachine readable instructions that may be executed to implement theexample demographic corrector of FIGS. 1 and 4 .

FIG. 10 is a flow diagram representative of example machine readableinstructions that may be executed to implement the example viewershipassigner of FIGS. 1 and 5 .

FIG. 11 is another flow diagram representative of example machinereadable instructions that may be executed to implement the exampleviewership assigner of FIGS. 1 and 5 .

FIG. 12 is a flow diagram representative of example machine readableinstructions that may be executed to implement the example demographicpredictor of FIGS. 1 and 6 .

FIG. 13 illustrates an example processor system structured to executethe example instructions of FIGS. 7, 8, 9A-9C, 10, 11, and 12 toimplement the example apparatus of FIG. 1 to correct demographicimpressions, assign viewership, and predict demographics frominformation obtained from OTT devices and database proprietors.

DETAILED DESCRIPTION

Examples disclosed herein correct demographic impression information forany type of media (e.g., content and/or advertisements) broadcastthrough over-the-top (“OTT”) devices.

As used herein, over-the-top refers to the delivery of audio, video,and/or other media over the Internet without a multiple-system operatorbeing involved in the control and/or distribution of the media. Forexample, over-the-top media is distributed from one or more thirdparties to an end user through an Internet-enabled device. Example OTTdevices include, without limitation, Roku® devices, Apple TV® devices,Samsung Smart TV® devices, Chromecast devices, Amazon® Fire TV devices,etc. While examples disclosed herein generally refer to OTT devices,such examples may apply to any media delivered via satellite, cabletelevision, radio frequency (RF) terrestrial broadcast, the Internet(e.g., internet protocol television (IPTV)), television broadcasts,radio broadcasts and/or any other type of transmission for deliveringmedia where little, none, or incorrect demographic impressioninformation exists.

As used herein, an impression is defined to be an event in which a homeor individual is exposed to corresponding media. Thus, an impressionrepresents a home or an individual having been exposed to media (e.g.,an advertisement, content, a group of advertisements, and/or acollection of content). In Internet media access, a quantity ofimpressions or impression count is the total number of times media(e.g., content, an advertisement or advertisement campaign) has beenaccessed by an Internet audience.

Media includes advertising and/or content. Example types of mediainclude web pages, text, images, streaming video, streaming audio,movies, and/or any other type of content and/or advertisements. In someexamples, media includes user-generated media that is, for example,uploaded to media upload sites such as YouTube® and subsequentlydownloaded and/or streamed by one or more client devices for playback.Advertisements are typically distributed with content (e.g.,programming). Traditionally, content is provided at little or no cost tothe audience because it is subsidized by advertisers that pay to havetheir advertisements distributed with the content. As used herein,“media” refers collectively and/or individually to content and/oradvertisement(s) of any type(s).

In examples disclosed herein, an impression is associated with thedemographics of the person corresponding to the impression to track aperson's exposure to media. As used herein, a demographic impression isdefined to be an impression that is associated with a characteristic(e.g., a demographic characteristic: age, gender, geographic location,race/ethnicity, income level, education level, religion, etc.) of theperson(s) exposed to the media. Mapping demographics to impressionsprovides, in some examples, measurements of media exposure and/oraudience behavior across numerous demographic groups (e.g., differentages, different genders, etc.). These numerous demographic groups (e.g.,different ages) are sometimes referred to as “demographic buckets.” Forexample, demographic buckets may be defined for ages 2-12, 13-14, 15-17,18-20, 21-24, 25-29, 30-34, 35-39, 40-44, 45-49, 50-54, 55-64, 65+ andfor each gender (e.g., male and female).

To associate demographics to an impression, the demographics of theperson corresponding to the impression (or someone with similardemographics) must be known. Some OTT devices employ a user-registrationmodel that may be used to obtain user demographics. As used herein, auser-registration model is a model in which users subscribe tomedia-delivery services by creating an account and providingdemographic-related information about themselves. For example, suchaccounts are created with manufacturers of the example OTT devicesand/or media service providers that enable media delivery to the exampleOTT devices. In some instances, OTT device user-registration modelsprovide only partial demographic data (e.g., age and/or email address)for a person or for a household (“HH”) (e.g., one person registers theexample OTT device and multiple people use the device). As such,user-registration models for OTT devices may not produce highly accuratedemographic information about audience member that access media via theexample OTT devices.

Often OTT devices provide media (e.g., content and/or advertising)without tracking exposures (e.g., impressions) to the media. As aresult, demographics are unknown for audiences exposed to the media.Prior techniques that collect impressions for online media accessesidentify associated demographic information by using locally-installedpanelist meter software on a personal computer and/or cookies toassociate demographic data to collected impressions.

Typically, OTT devices do not operate with cookies. Additionally, OTTdevices are not easily configurable for the installation and running ofpanelist meter software. In some examples, OTT devices havecorresponding identification, such as OTT device identifiers (“IDs”),which can be used to collect demographic impressions by associatingimpressions with demographics known to correspond to the example OTTdevice IDs (e.g., based on OTT service user account information.However, if such device IDs are blocked, changed, or otherwise obscured(e.g., by users of the example OTT devices), demographics would not beassociated with impressions because the obscured OTT device IDs couldnot properly identify such demographics. As such, current techniquesarbitrarily spread impressions from OTT devices with obscured OTT deviceIDs across various demographic buckets in an attempt to associatedemographics with such impressions.

Examples disclosed herein can be used to increase the demographicaccuracy of demographic impressions corresponding to OTT devices.Examples disclosed herein leverage panelist information and demographicinformation collected by database proprietors to increase the accuracyof demographic impressions associated with OTT devices. For example,database proprietors gather detailed demographics (e.g., age, gender,geographic location, race, income level, education level, religion,etc.) of their subscribers during user registrations. Databaseproprietor, as used herein, refers to any entity having anetwork-accessible database storing and/or maintaining records orprofiles identifying demographics of users having accounts therewith.Such database proprietors include, for example, social networking sites(e.g., Facebook®, Twitter®, MySpace®, etc.), multi-service providers(e.g., Google®, Yahoo!®, MSN®, Apple iTunes®, etc.), media deliveryservice providers (e.g., service providers that provide media deliveryservices for Roku® devices, Apple TV® devices, Google® TV devices,Chromecast devices, Samsung Smart TV® devices, Amazon® Fire TV devices,etc.), online retailer sites (e.g., Amazon.com, Buy.com, etc.), creditreporting services (e.g., Experian®), and/or any other web service(s)site that maintains user registration records.

In the examples of social networking sites (e.g., Facebook®, Google+®,etc.), user demographics are plentiful because users create detailedprofiles to represent themselves online. In some instances, peopleregistered with a database proprietor are also registered with OTTdevice service providers. In some examples, the database proprietor andthe example OTT device service provider have the same or substantiallysimilar demographic data for the same person and/or household registeredwith both services. In other examples, database proprietors and OTTdevice service providers have significantly different demographic datafor the same person.

In examples disclosed herein, demographics corresponding to registeredusers of an OTT device (or other similar device) are compared todemographics corresponding to registered users of database proprietorsto identify matches. In some examples disclosed herein, no matches occurbetween OTT devices (or other similar devices) and database proprietors.This may occur because there is little to no demographic data (e.g.,age, gender, etc.) available for a registrant through an OTT device. Forexample, the example OTT device registration process may not require thedemographic inputs for age or gender to register for media deliveryservices. Additionally or alternatively, there may be little to nodemographic data (e.g., age, gender, etc.) available for the registrantthrough a database proprietor. In some examples, an OTT deviceregistrant is not associated with any database proprietors. In someexamples, demographic data that exists for the registrant through an OTTdevice contradicts demographic data available through a databaseproprietor. Such circumstances can lead to impressions with unknowndemographics when an audience measurement entity processes impressionsfrom OTT devices to associate them with demographics from databaseproprietors. For example, impressions collected based on media accessedthrough OTT devices are associated with no one in particular or arelabeled as “unknown” because they do not match any user registered witha database proprietor. Empty demographics or non-attributions ofdemographics to impressions lead to gaps in overall audience metrics,which decreases the accuracy of audience measurements.

In other examples disclosed herein, there are matches betweendemographics corresponding to the example OTT device registrants anddemographics corresponding to database proprietor registrants. Forexample, a match occurs when a user uses the same or substantiallysimilar basic demographic information (e.g., age and gender) to registeracross multiple services such as an OTT device service and a databaseproprietor service. In other examples, more detailed demographics (e.g.,personal income, viewing habits, social security number, credit cardnumber, etc.) are required across user registrations to confirm a matchindicative of a same person registered with, for example, the exampleOTT device service and the database proprietor. By matching demographicsfrom an OTT device with demographics from database proprietors, examplesdisclosed herein combine, correct, and/or adjust demographics from OTTdevice services and/or database proprietors to generate more accurateand/or more detailed demographics and demographic impressions.

In some instances, demographics from OTT devices, database proprietors,and/or other demographic databases are not always accurate, even whenmatches are used to combine demographics from OTT device services anddatabase proprietors. Misattribution errors (e.g., designatingimpressions with incorrect demographics) can sometimes occur fordemographic impressions, such as, for example, where the actual age ofthe person corresponding to the impression is different that than thereported age of the person. For example, a potential subscriber to adatabase proprietor (e.g., Facebook®, Yahoo! ®, Google®, Experian®,etc.) is prevented from registering therewith if he or she is below arequired age (e.g., thirteen, eighteen, twenty-one, etc.) or alternatedemographic requirement. Accordingly, in some examples, the potentialsubscriber often provides false, but required, conforming demographicinformation to register. Consequently, impressions associated with thedemographics of the example subscriber from the example databaseproprietor potentially will be misattributed to an incorrect demographicgroup. In some examples, misattribution errors decrease the accuracy ofaudience measurements.

Additionally, while a first user having a first demographic in ahousehold registers an OTT device and has associated demographic datawithin a database proprietor, another user (e.g., a second user, thirduser, a fourth user, etc.) with a different demographic in the samehousehold may use the same OTT device as the first user. Thus, thedemographics of the actual user will not match those of householdmembers who register the OTT device. In some examples, the example OTTdevice has a device identification (ID) and user profiles. However,impressions are often linked to the device ID and not the user profiles,because the user profiles do not require separate registration (andtherefore, demographic data entry). For example, in households withmultiple people, more than one person may share the example OTT device(and thus, its device ID) to access media without providing anindication of which member of the household is using the device (e.g.,an 8-year old male child using the example OTT device ID registered tohis 35-year old mother). In such examples, the impression data generatedwhile the 8-year old male child was using the example OTT device wouldbe misattributed to the 35-year old mother. As such, impressions can bemisattributed to the wrong household member. Such misattributions reducethe accuracy of audience measurement.

To improve accuracies of demographic impressions and audiencemeasurement in general, examples disclosed herein obtain household-levelmatched impressions from OTT devices and database proprietors, correctmisattribution errors in matched impressions, predict the demographicsof non-matched impressions, and assign members of a household toparticular impressions.

Traditionally, audience measurement entities (also referred to herein as“AME” and/or “ratings entities”) determine demographic reach, exposure,and other data for advertising and media programming based on registeredpanel members. That is, an audience measurement entity enrolls peoplewho consent to being monitored into a panel. During enrollment, theaudience measurement entity receives demographic information from theenrolling people so that subsequent correlations are made between mediaexposures to those panelists and different demographic markets. Often,the panelist data is the most accurate, but also the smallest in samplesize. Unlike traditional techniques in which audience measuremententities rely solely on their own panel member data to collectdemographics-based audience measurements, example methods, apparatus,and/or articles of manufacture disclosed herein compare demographicinformation stored by an audience measurement entity with demographicinformation from other entities (e.g., database proprietors) thatoperate based on user-registration models.

Examples disclosed herein apply panelist data to demographicsinformation from external sources (e.g., database proprietors, OTTdevices, etc.), to extend or supplement the coverage, accuracy, and/orcompleteness of an AME's demographics-based audience measurements.Examples disclosed herein provide the AME with monitoring data (e.g.,demographic impressions) for people are not part of or who would nototherwise have joined an audience measurement panel.

To increase the likelihood that measured viewership is accuratelyattributed to the correct demographics, examples disclosed herein usedemographic information stored by the audience measurement entity aswell as demographic information stored by one or more databaseproprietors. Examples disclosed herein supplement demographicinformation maintained by a ratings entity (e.g., an AME such as TheNielsen Company of Schaumburg, Ill., United States of America, thatcollects media exposure measurements and/or demographics) withdemographic information from one or more different database proprietors.Additionally, examples disclosed herein supplement and/or correctdemographic information maintained by the one or more different databaseproprietors with demographic information from the ratings entity.

The use of demographic information from disparate data sources (e.g.,high-quality demographic information from the panels of an audiencemeasurement company and/or registered user data of web serviceproviders) provides improved reporting effectiveness of metrics for bothonline and offline advertising campaigns and/or media (e.g., downloadedand/or streamed video and/or audio media). Examples disclosed herein useregistration data from OTT device registrants, database proprietorregistrants, and AME panelists to correct demographic impressions of OTTdevice users, to predict the demographics of missing members of ahousehold, and to assign members of a household to particularimpressions.

Examples disclosed herein identify AME panelists who are alsoregistrants of a database proprietor and an OTT device. Examplesdisclosed herein analyze panelist demographic information as submittedto the AME Panel database against panelist demographic informationsubmitted to the database proprietor and OTT device service provider. Inthe illustrated example, the AME panelist demographics are used tocorrect the database proprietor demographics of such AME panelists.Examples disclosed herein build a model based on the corrections.Examples disclosed herein apply the example model to the entirety of thedatabase proprietor's demographic information (e.g., non-panelistdemographic information) to correct deficiencies in the databaseproprietor's demographics.

Examples disclosed herein additionally assign viewership to particulardemographic impressions and predict demographics missing or unknown inthe database proprietor's demographics. In some examples, assigningviewership may utilize methods and apparatus disclosed in U.S. patentapplication Ser. No. 14/313,390 entitled “METHODS AND APPARATUS TOCHARACTERIZE HOUSEHOLDS WITH MEDIA METER DATA” filed on Jun. 24, 2014,which is hereby incorporated herein by reference in its entirety.

An example method includes identifying, via a processor, a first set ofdata from a first data source, the first set of data different from asecond set of data from a second data source, generating, via theprocessor, a third set of data based on the second set of data from thesecond data source, generating, via the processor, a model based on adifference between the first set of data and the third set of data,applying, via the processor, the model to the first set of data, andassigning, via the processor, viewership to an impression associatedwith the first set of data by imputing viewership associated with thesecond set of data to the first set of data.

In some examples, the second set of data is data from panelists enlistedin audience measurement.

In some examples, the generating of the third set of data includesidentifying a demographic bucket, determining whether the demographicbucket exists in a household based on an independent binary model, anddetermining the number of members in the household associated with thedemographic bucket based on a demographic distribution associated withthe second set of data.

In some examples, the independent binary model is a machine learningalgorithm generated based on at least one of a household size, age,gender, person status, income, education, or ethnicity.

In some examples, the machine learning algorithm is at least one of aclassification and regression tree (“CART”), logit, conditionalinference tree (“ctree”), random forest, neural networks, or bootstrapaggregate decision trees.

In some examples, the assigning of the viewership to the impressionincludes, identifying viewing history associated with the second set ofdata, determining a first time associated with a first demographicviewing a media presentation in a first household associated with thesecond set of data, determining a second time associated with the firstdemographic and a second demographic viewing the media presentation inthe household, determining a first probability that the firstdemographic viewed the media presentation by dividing the first time bythe second time, identifying a first person in the first householdassociated with the second set of data having a second probabilitysimilar to the first probability, and imputing a viewing history of thefirst person to a second person in a second household associated withthe first set of data.

In some examples, the identifying of the first person in the firsthousehold associated with the second set of data having the secondprobability similar to the first probability includes identifying athird person in the first household associated with the second set ofdata, ranking the first and third person based on a first highestprobability, identifying a fourth person in the second householdassociated with the first set of data, ranking the second and fourthperson based on a second highest probability, and identifying the firstperson has the second probability similar to the first probability whenthe first person has the first highest probability and the second personhas the second highest probability.

In some examples, the first set of data from the first data source has afourth set of data missing, further including linearly scaling the firstset of data to accommodate for the missing fourth set of data.

An example apparatus includes a demographic corrector to identify afirst set of data from a first data source, the first set of datadifferent from a second set of data from a second data source, generatea third set of data based on the second set of data from the second datasource, generate a model based on a difference between the first set ofdata and the third set of data, apply the model to the first set ofdata, and a viewership assigner to assign viewership to an impressionassociated with the first set of data by imputing viewership associatedwith the second set of data to the first set of data.

In some examples, the second set of data is data from panelists enlistedin audience measurement.

In some examples, the demographic corrector is to identify a demographicbucket, determine whether the demographic bucket exists in a householdbased on an independent binary model, and determine the number ofmembers in the household associated with the demographic bucket based ona demographic distribution associated with the second set of data.

In some examples, the independent binary model is a machine learningalgorithm generated based on at least one of a household size, age,gender, person status, income, education, or ethnicity.

In some examples, the machine learning algorithm is at least one ofCART, logit, ctree, random forest, neural networks, or bootstrapaggregate decision trees.

In some examples, the viewership assigner is to identify viewing historyassociated with the second set of data, determine a first timeassociated with a first demographic viewing a media presentation in afirst household associated with the second set of data, determine asecond time associated with the first demographic and a seconddemographic viewing the media presentation in the household, determine afirst probability that the first demographic viewed the mediapresentation by dividing the first time by the second time, identify afirst person in the first household associated with the second set ofdata having a second probability similar to the first probability, andimpute a viewing history of the first person to a second person in asecond household associated with the first set of data.

In some examples, the viewership assigner is to identify a third personin the first household associated with the second set of data, rank thefirst and third person based on a first highest probability, identify afourth person in the second household associated with the first set ofdata, rank the second and fourth person based on a second highestprobability, and identify the first person has the second probabilitysimilar to the first probability when the first person has the firsthighest probability and the second person has the second highestprobability.

In some examples, the first set of data from the first data source has afourth set of data missing from the first set of data, further includinga demographic predictor to linearly scale the first set of data toaccommodate for the missing fourth set of data.

An example tangible computer readable storage medium includesinstructions that, when executed, cause a machine to at least identify afirst set of data from a first data source, the first set of datadifferent from a second set of data from a second data source, generatea third set of data based on the second set of data from the second datasource, generate a model based on a difference between the first set ofdata and the third set of data, apply the model to the first set ofdata, and assign viewership to an impression associated with the firstset of data by imputing viewership associated with the second set ofdata to the first set of data.

In some examples, the second set of data is data from panelists enlistedin audience measurement.

In some examples, the instructions, when executed, further cause themachine to identify a demographic bucket, determine whether thedemographic bucket exists in a household based on an independent binarymodel, and determine the number of members in the household associatedwith the demographic bucket based on a demographic distributionassociated with the second set of data.

In some examples, the independent binary model is a machine learningalgorithm generated based on at least one of a household size, age,gender, person status, income, education, or ethnicity.

In some examples, the machine learning algorithm is at least one ofCART, logit, ctree, random forest, neural networks, or bootstrapaggregate decision trees.

In some examples, the instructions, when executed, further cause themachine to identify viewing history associated with the second set ofdata, determine a first time associated with a first demographic viewinga media presentation in a first household associated with the second setof data, determine a second time associated with the first demographicand a second demographic viewing the media presentation in thehousehold, determine a first probability that the first demographicviewed the media presentation by dividing the first time by the secondtime, identify a first person in the first household associated with thesecond set of data having a second probability similar to the firstprobability, and impute a viewing history of the first person to asecond person in a second household associated with the first set ofdata.

In some examples, the instructions, when executed, further cause themachine to identify a third person in the first household associatedwith the second set of data, rank the first and third person based on afirst highest probability, identify a fourth person in the secondhousehold associated with the first set of data, rank the second andfourth person based on a second highest probability, and identify thefirst person has the second probability similar to the first probabilitywhen the first person has the first highest probability and the secondperson has the second highest probability.

In some examples, the first set of data from the first data source has afourth set of data missing from the first set of data, further includinga demographic predictor to linearly scale the first set of data toaccommodate for the missing fourth set of data.

FIG. 1 illustrates an example system 100 with an example over-the-top(“OTT”) device 102, an example database proprietor (“DP”) 104 having aDP database 105, and an example audience measurement entity (“AME”) 106.An example demographic comparator 107 communicates with the example OTTdevice 102 (and/or an OTT device service provider of the example OTTdevice 102) and the example database proprietor 104 and/or DP database105 to create demographic impressions, as further discussed inconjunction with FIGS. 2 and 3 .

The example demographic comparator 107 compares demographic informationfrom the example database proprietor 104 with information identified bythe example OTT device 102 (e.g., information sent to a proprietor ofthe example OTT device 102 during, for example, a registration process)to identify matches and create matched demographics. In some examples,the example demographic comparator 107 matches person-level information(e.g., identifiers, demographics, etc.) from the example databaseproprietor 104 to person-level information received from the example OTTdevice 102 (e.g., during registration) based on person identifiers(e.g., name, date of birth, address, etc.). In such examples, theexample demographic comparator 107 creates person-level matcheddemographics. In examples wherein person identifiers are unavailable(e.g., due to privacy concerns), the example demographic comparator 107matches household-level information (e.g., identifiers, demographics,etc.) from the example database proprietor 104 to household-levelinformation received from the example OTT device 102 based on ahousehold identifier. For example, the example demographic comparator107 associates demographic information associated with a householdidentifier from the example database proprietor 104 with a samehousehold identifier from the example OTT device 102. In such examples,the example demographic comparator 107 creates household-level matcheddemographics. However, in some examples, the demographics from theexample database proprietor 104 is different that the demographics fromthe example OTT device 102 (e.g., the example database proprietor 104identifies a male having an age between 18 and 20, whereas the exampleOTT device 102 identifies a male between the age of 13 and 14), eventhough the household is matched via the household identifier. Therefore,in some examples, the matched demographics form the example demographiccomparator 107 need to be corrected.

To find matching data between the example database proprietor 104 and aproprietor of the example OTT device 102 on a person-level, the exampledemographic comparator 107 determines whether a first data element(e.g., name, email address, credit card number, etc.) identified duringregistration of the example OTT device 102 matches with a second dataelement (e.g., name, email address, credit card number, etc.) identifiedin the example DP database 105. If the first data element matches thesecond data element, then the example demographic comparator 107combines (e.g., via data fusion techniques) all data elements receivedduring registration of the example OTT device 102 and all data elementsidentified in the example DP database 105 to create matcheddemographics. In some examples, the example demographic comparator 107generates a table (e.g., a column with OTT device 102 registrationinformation and a column with information from the example DP database105).

To find matching data between the example database proprietor 104 and aproprietor of the example OTT device 102 on a household-level, theexample demographic comparator 107 determines whether a first dataelement (e.g., OTT device ID) identified during registration of theexample OTT device 102 is associated with a second data element (e.g., ahousehold ID) identified in the example DP database 105. If the firstdata element matches the second data element, then the exampledemographic comparator 107 combines (e.g., via data fusion techniques)all data elements received during registration of the example OTT device102 and all data elements identified in the example DP database 105 tocreate matched demographics. In some examples, the example demographiccomparator 107 generates a table (e.g., a column with OTT device 102registration information and a column with information from the exampleDP database 105). The example demographic comparator 107 may matchdemographics before or after the impression collection phase, asdescribed herein. The example demographic comparator 107 sends thematched demographics (e.g., either person-level matched orhousehold-level matched) to an example apparatus 108. In some examples,the matched demographics are associated with impressions collectedduring the impression collection phase.

In some examples, the example demographic comparator 107 does notidentify information from the example DP database 105 that matches withinformation received during registration of the example OTT device 102(e.g., the registrant of the example OTT device 102 may not beassociated with the example database proprietor 104, the registrant ofthe example OTT device 102 may have provided false information whenregistering for at least one of the example OTT device 102 or theexample database proprietor 104, etc.). In such examples, the exampledemographic comparator 107 sends this non-matched demographic data tothe example apparatus 108. In some examples, the non-matched demographicdata are associated with impressions collected during the impressioncollection phase.

The example AME 106 operates the example apparatus 108 to correct anydifferences in demographics associated with impressions. The exampleapparatus 108 also assigns viewership for the impressions and predictsdemographics from information obtained from over-the-top devices anddatabase proprietors where such demographics are not provided (e.g.,unknown). The example apparatus 108 includes an example demographiccorrector 110, an example viewership assigner 112, an exampledemographic predictor 114, an example panelist database 116, and anexample aggregator 118. In the illustrated example, the exampleapparatus 108 generates reports 120 based on the example demographiccorrector 110, the example viewership assigner 112, and the exampledemographic predictor 114. The example OTT device 102, an exampledatabase proprietor 104, the example AME 106, the example apparatus 108,the example demographic corrector 110, the example viewership assigner112, and the example demographic predictor 114 are all connected to anexample network 122 in the illustrated example.

The example demographic corrector 110 receives matched demographics fromthe example demographic comparator 107. Depending on whether thedemographic data is matched at a household-level or a person-level, theexample demographic corrector 110 corrects misattributed demographics inthe received demographic data. When the demographic data is matched at ahousehold-level, the example demographic corrector 110 first predictsthe actual demographics (e.g., demographic buckets) that exist within ahousehold (e.g., is there at least one person in the household that is amale between ages 18 and 20). Then, the example demographic corrector110 predicts the number of people having the predicted demographicswithin the household (e.g., how many people are in the household thatare male between ages 18 and 20) based on conditional probabilities frompanelist data. For example, a conditional probability is a probabilityof a person or persons in a household being a particular demographic(e.g., age, gender, etc.) based on a probability that there is at leastone person in the household being that particular demographic (e.g.,probability there are two females over the age of 65 given that there isat least one female over the age of 65). The example demographiccorrector 110 corrects the received matched demographic data based thepredicted demographics. In some examples, any differences in thereceived matched demographic data are adjusted to match the predicteddemographics. This is considered a household-level correction.

For a person-level correction based on a household-level match, theexample demographic corrector 110 matches household members to panelisthousehold members based on age and gender and corrects the receiveddemographic data based on the differences between the household membersand panelists. Also, when the demographic data is matched at aperson-level, the example demographic corrector 110 determines missingand extra household members. In some examples, the example demographiccorrector 110 corrects demographic information on a quarterly basis(e.g., every three months). Of course, the example demographic corrector110 may correct demographic information at different frequencies (e.g.,daily, weekly, monthly, etc.).

The example viewership assigner 112 obtains household impressions andcorrected demographics from the example demographic corrector 110 and/orthe example network 122. In the illustrated example, the exampleviewership assigner 112 obtains donor household viewing history (e.g.,panelist viewing history) including probabilities of particular panelistviewers based on demographics. The example viewership assigner 112identifies probabilities that users having the demographics corrected bythe example demographic corrector 110 viewed a media presentationassociated with the household impression. In some examples, the exampleviewership assigner 112 merges the panelist probabilities with thehousehold impression data to obtain probabilities for each user in thehousehold. The example viewership assigner 112 ranks the probabilitiesof the users and the probabilities of the panelist viewers to determinedonor viewing events. A viewing event may be consecutive viewing minutes(e.g., 1-421 minutes) of the same program by the same people in ahousehold. In some examples, the example viewership assigner 112 mergesimpression data with a table of possible donor viewing events, whichprovides a list of all possible donors for each OTT viewing event.Thereafter, the example viewership assigner 112 identifies a donorviewing event based on panelist donors having similar probabilities tothe users. If there is only one possible match, the example viewershipassigner 112 identifies that viewing event as the donor. If there ismore than one possible match, the example viewership assigner 112 usesthe difference in probabilities to determine the match. If there is morethan one viewing event with the same minimum difference inprobabilities, then the example viewership assigner 112 randomly selectsa viewing event. Alternatively, the most recent viewing event or aviewing event associated with panelists having demographic compositionssimilar to the users may be chosen. Finally, the example viewershipassigner 112 assigns viewership by ranking users within the householdand imputes the viewing histories (e.g., viewed or not viewed) ofpanelist donors having a similar rank. In some examples, all users in ahousehold are assigned viewership, and the example viewership assigner112 determines whether the media presented associated with theimpression conflicts with a user to disqualify that user from viewership(e.g., the user normally doesn't watch the media, the user doesn't watchany media during the time the media was presented, etc.).

The example demographic predictor 114 predicts demographic data toaccount for the non-matched demographic data received from the exampledemographic comparator 107. In some examples, the demographic predictor114 linearly scales the corrected demographics from the exampledemographic corrector 110. In some examples, the example demographicpredictor 114 linearly scales demographic data matched by the exampledemographic comparator 107 and sends the scaled demographics to theexample demographic corrector 110. Additionally or alternatively, theexample demographic predictor 114 applies a differential scaling factor(e.g., a scaling factor that is different across numerous demographicbuckets).

In operation, the example demographic comparator 107 matches demographicinformation from the example database proprietor 104 with informationreceived from the example OTT device 102 (e.g., during registration). Insome examples, such information is stored by a proprietor of the exampleOTT device 102 in association with the example OTT device ID or otheridentifier. The example demographic comparator 107 creates a matched IDto associate the matched demographics with impressions of mediapresented through the example OTT device 102. The example demographiccorrector 110 obtains matched demographics and/or impressions from theexample demographic comparator 107. To increase the accuracy of thematched demographics, the example demographic corrector 110 leveragesthe example panelist database 116. The example demographic corrector 110utilizes panelist information to correct misattributions in the matcheddemographics. The corrected demographics and the respective impressionsare sent to the example viewership assigner 112. The example viewershipassigner 112 associates particular household members with the impression(e.g., whether a particular household member viewed a particular mediapresentation). The example demographic predictor 114 scales or otherwisepredicts non-matched demographic data from the example demographiccomparator 107.

After the demographics have been corrected by the example demographiccorrector 110, the viewership has been assigned by the exampleviewership assigner 112, and the unknown demographics are predicted bythe example demographic predictor 114, the results are aggregatedtogether by the example aggregator 118. Subsequently, the exampleapparatus 108 generates and/or exports a plurality of reports 120 withthe assigned viewership and corrected/predicted demographics, to reportdemographic impressions with relatively higher accuracies from OTTdevices.

FIG. 2 illustrates an example OTT registration phase 200 to generate anidentifier for the example OTT device 102 and associate the identifierwith demographics of a user that accesses media via the example OTTdevice 102. As shown in FIG. 2 , the example OTT device 102 sends anexample OTT device ID 202 (e.g., A35Z) along with a first set of data,such as, for example, user data 204, to both the example demographiccomparator 107 and the example database proprietor 104. Examples of theexample OTT device ID 202 include, without limitation, a device serialnumber, a unique hardware identifier (e.g., a Roku® device ID), or otherknown identifier that uniquely identifies the example OTT device 102.The example user data 204 includes a user's name, email address, creditcard number, OTT device ID, a matched ID (FIG. 3 ), and/or otherinformation required during registration of the example OTT device 102.In the illustrated example, the database proprietor 104 matches theexample user data 204 to a second set of data, such as, for example,demographics 206 from the example DP database 105. The exampledemographics 206 include similar and/or different information about theuser such as name, age, email address, education, physical address,credit card number, gender, household ID, matched ID (FIG. 3 ), and/orother information provided during registration for and/or use of aservice provided by the example database proprietor 104 (e.g.,Facebook®, Experian®, etc.). In some examples, the database proprietor104 matches the example user data 204 to the example demographics 206using any common data elements (e.g., name, email address, credit cardnumber, matched ID, etc.) and combines, merges, or otherwise utilizesdata fusion to supplement the example user data 204 with the exampledemographics 206, to supplement the example demographics 206 with theexample user data 204, form matched demographics combining the exampleuser data 204 with the example demographics 206, etc. In some examples,the example OTT device 102 communicates directly with the exampledatabase proprietor 104, and the example database proprietor 104compares the demographics 206 to the example OTT device user data 204and/or the example OTT device ID 202. In yet other examples, the exampleOTT device 102 does not communicate with the example database proprietor104, and the example database proprietor 104 sends demographics to theexample demographic comparator 107, which performs the demographiccomparisons as described herein.

In the illustrated example, the example user data 204 from the exampleOTT device 102 includes user registration information (e.g., an OTTdevice ID, a name, an address, an age, a gender, a credit card number,any combination thereof, or any other identifying information). In theillustrated example, the example demographics 206 from the exampledatabase proprietor 104 includes user registration/profile informationpertaining to a name, an address, an age, a gender, an education, areligion, any combination thereof, or any other identifying informationif the user is a subscriber of the example database proprietor 104. Insome examples, the example database proprietor 104 associates users thatare members of the same household together into a household. In suchexamples, the example database proprietor generates an example databaseproprietor household identifier (“DP HE ID”) to identify the household.In the illustrated example, the example database proprietor 104 sendsthe example demographics 206 of the user to the example demographiccomparator 107.

The example database proprietor 104 is one of many database proprietorsthat operate on the Internet and provide services to large numbers ofsubscribers. Such services may be email services, social networkingservices, news media services, cloud storage services, streaming musicservices, streaming video services, online retail shopping services,credit monitoring services, etc. In examples disclosed herein, theexample database proprietor 104 maintains user account recordscorresponding to users registered for Internet-based services providedby the database proprietors. That is, in exchange for the provision ofservices, subscribers register with the example database proprietor 104.As part of this registration, the subscribers provide detaileddemographic information to the example database proprietor 104.Demographic information may include, for example, gender, age,ethnicity, income, home location, education level, occupation, etc.

The example demographic comparator 107 receives or otherwise obtains theexample user data 204 from the service provider of the example OTTdevice 102 (or the example OTT device 102 itself). Additionally, theexample demographic comparator 107 receives or otherwise obtains theexample demographics 206 from the example database proprietor 104. Inexamples disclosed herein, the example demographic comparator 107generates an example secure, encrypted matched ID 208 (e.g., 3AZ7) toassociate demographics received from the example OTT device 102 anddemographics stored by the example database proprietor 104. When commondata elements are identified, the example demographic comparator 107matches, combines, or otherwise associates the user data 204 with thedemographics 206 to create matched demographics 210. In some examples,the example demographic comparator 107 sends the matched demographics210 with the example matched ID 208 (e.g., 3AZ7) to the example OTTdevice 102 and the example database proprietor 104.

In the illustrated example, the example demographic comparator 107identifies matches between the example user data 204 and the exampledemographics 206 at a household-level. For example, the exampledemographic comparator 107 determines whether a data element (e.g., OTTdevice ID) of the example user data 204 from the example OTT device 102matches a data element (e.g., DP HH ID) of the example demographics 206from the example database proprietor 104. If the data element from theexample user data 204 from the example OTT device 102 matches the dataelement from the example demographics 206 from the example databaseproprietor 104, the example demographic comparator 107 generates theexample secure, encrypted matched ID 208 linking the example user data204 from the example OTT device 102 to the example demographics 206 fromthe example database proprietor 104. The aforementioned matching processis used to create matched demographic impressions as disclosed herein.While the matching is described in conjunction with matching the OTTdevice ID to a DP HH ID, other data elements may be used to matchhouseholds and/or users.

In some examples, the example demographic comparator 107 sends theexample matched ID 208 to the example OTT device 102, the exampledatabase proprietor 104, and the example demographic corrector 110. Insome examples, the example demographic comparator 107 associates theexample matched ID 208 with the corresponding example matcheddemographics 210 because the example OTT device ID 202 is capable ofbeing changed, reset, hidden, or otherwise altered to prevent accuratemeasurement and/or demographic impression association. Thus, the exampleOTT device 102 and the example demographic corrector 110 rely upon theexample matched ID 208 for accurate measurement and demographicimpression association.

If the example user data 204 from the example OTT device 102 does notmatch (e.g., there are no common data elements) the example demographics206 from the example database proprietor 104, the example demographiccomparator 107 sends the example user data 204 and the exampledemographics 206 to the example demographic predictor 114 (FIG. 1 ). Forexample, the example demographic comparator 107 may have a match rate of73% for 17,552 households. In such examples, the non-matching occursbecause the example database proprietor 104 does not have data withwhich to compare to the user data 204 (e.g., the user of the example OTTdevice 102 is not associated with the example database proprietor 104).In the illustrated example, the example demographic predictor 114predicts or otherwise obtains demographic data for unmatched data byscaling matched impressions from the example demographic comparator 107to accommodate non-matched impressions. In some examples, demographicimpressions are matched to particular users via IP addresses of OTTdevices of those users (or via IP addresses of a household router,modem, etc.). In such examples, a database proprietor (e.g., eXelate)provides demographics for a particular user based upon the user's IPaddress. IP demographic matching may occur when the example demographiccomparator 107 cannot match the user data 204 to the demographics 206.Alternatively, IP demographic matching occurs in parallel with, orinstead of, the example demographic comparator 107. In some examples,the example demographic comparator 107 is operated by the databaseproprietor 104. In other examples, the example demographic comparator107 is operated by the example AME 106. In yet other examples, theexample demographic comparator 107 is operated by a third party.

The example registration phase of FIG. 2 associates an identifier 202 ofthe example OTT device 102 (e.g., a client device) with demographics ofa user that accesses media via the example OTT device 102. The user maybe a head of household, or a person likely to register a device for thehousehold. The example AME 106 identifies a likely profile of thehousehold and/or demographics of other members living in the samehousehold based on the demographics of the user. In examples disclosedherein, a household composition for the household of the user aremodeled based on the demographics of the head of household (e.g.,demographics of the user), media access characteristics (e.g., name,genre, volume level, application name, application category, time of dayof exposure, database household demographics), and models based on knownpanel family compositions indicated in panel data as likely to access aparticular mix of media (e.g., TV shows and/or other programming) thatsubstantially matches the media accessed on the example OTT device inthe household of the user.

In some examples, a user registers the example OTT device 102 through anexample OTT registration website to access one or more media deliveryservices via the example OTT device 102. In some examples, when the userconfigures the example OTT device 102 for use, the example OTT device102 displays example OTT device registration instructions on an examplemedia presentation device. The example OTT device 102 instructs the userto navigate to the example OTT registration website using a computer, oralternatively through the example OTT device 102 itself. The example OTTdevice 102 also provides to the user the example OTT device identifierID 202 (e.g., A35Z) that the user is to enter during registration. Whenthe user enters the example OTT device ID 202 during the example OTTregistration, the manufacturer or service provider of the example OTTdevice 102 links the example OTT device 102 with a user account of theuser based on the example OTT device ID 202. By making this associationbetween the particular OTT device 102 and a user account, the user isable to set up media-streaming services on the example OTT device 102and make purchases via the example OTT device 102. Further, themanufacture or provider of the example OTT device 102 obtains knowledgeof who is associated with each OTT device ID 202.

To monitor media accessed via the example OTT device 102, an example AME104 requires a static identifier (e.g., an identifier that does notchange), with which the example OTT device ID 202 fails to conform, insome examples. Therefore, in some examples, the example demographiccomparator 107 collects and stores the example OTT device ID 202 of theexample OTT device 102 during the example registration phase of FIG. 2 .However, because the example OTT device ID 202 is capable of changing,the example demographic comparator 107 creates the example matched ID208 to be associated with the example OTT device 102. In the illustratedexamples, the example demographic comparator 107 uses the examplematched ID 208 to associate the demographics 206 received from theexample database proprietor 104 and/or the user data 204 received fromthe example OTT device 102 (e.g., matched demographics 210) with theexample OTT device 102 and, thus, impressions subsequently received fromthe example OTT device 102. In some examples, the example matched ID 208is the same as the example OTT device ID 202.

In some examples, the example demographic comparator 107 obfuscates theexample OTT device ID 202 from the example database proprietor 104. Insuch examples, the example demographic comparator 107 generates a valuefor the example matched ID 208 that is different from the example OTTdevice ID 202. In some examples, the example matched ID 208 may be ahash of the example OTT device ID 202. Of course, as mentionedpreviously, the example demographic comparator 107 may be part of theexample AME 106, the example database proprietor 104, and/or a thirdentity.

As shown in FIG. 1 , the example demographic corrector 110 has access tothe example panelist database 116, wherein the example AME 106 canassociate the example matched ID 208 with panelist demographicspreviously collected by the example AME 106 from the user, if the useris a panelist of the example AME 106. For example, the example AME 106establishes a panel of users who have agreed to provide theirdemographic information and to have their OTT media access activitiesmonitored. When an individual joins the AME panel, the person providesdetailed information concerning the person's identity and demographics(e.g., gender, age, ethnicity, income, home location, occupation, etc.)to the example AME 106. The example AME 106 sets a device/useridentifier (e.g., a AME cookie) on the person's television set top box,computer, OTT device, etc. that enables the example AME 106 to identify,for example, the user (e.g., a panelist). An AME panel may be across-platform home television/computer (TVPC) panel built andmaintained by the example AME 106. In other examples, the AME panel is acomputer panel or internet-device panel without corresponding to atelevision audience panel. In yet other examples, the AME panel may be across-platform radio/computer panel and/or a panel formed for othermediums.

Although some examples disclosed herein are described in connection withAME panel members and corresponding panelist demographics, in someexamples, none or only some of the audience members of OTT devices arepanel members of an AME panel. Examples disclosed herein are useful tocollect demographics from database proprietors such as the exampledatabase proprietor 104 even when an audience member is not a panelmember of the example AME 106. In the illustrated example, the exampleAME 106 receives subscriber demographics from an example DP 104 asdisclosed herein without the example AME 106 also supplying AME panelistdemographics. In some examples, the example AME 106 does not provide AMEpanelist demographics for impressions, and collects demographics onlyfrom database proprietors.

FIG. 3 illustrates an example impression collection phase 300 to collectdemographic impressions corresponding to the example OTT device of FIGS.1-2 . In the illustrated example of FIG. 3 , the example OTT device 102receives media from media providers via the Internet over an internetprotocol (IP) connection. In the illustrated example, the example OTTdevice 102 may be any IP-based media delivery device capable ofreceiving, decoding, and presenting video, audio, and/or images. ExampleOTT devices include a Roku® media device, an Apple TV® media device, aSamsung Smart TV® media device, a Google® TV media device, a Chromecastmedia device, an Amazon® TV media device, a gaming console (e.g., aMicrosoft® Xbox® gaming console, a Sony® PlayStation® gaming console, aNintendo® Wii® gaming console, etc.), a smart TV, a smart DVD player, anaudio-streaming device, etc.

In the illustrated example, the example media is requested by anaudience member 302 (e.g., an OTT device subscriber) through a mediapresentation device 304. In the illustrated example, the AME 106 and/orthe example database proprietor (DP) 104 identify a likely profile ofthe household and/or demographics of other members living in the samehousehold based on the demographics of the audience member 302. In someexamples, a household composition for the household of the audiencemember 302 is modeled based on the demographics of the head of household(e.g., demographics of the audience member 302), media accesscharacteristics (e.g., genre and/or volume of media being accessed inthe household), and/or known panel family compositions indicated inpanel data as likely to access a particular mix of media (e.g., TV showsand/or other programming) that substantially matches the media accessedin the household of the audience member 302.

The example media presentation device 304 is connected to the exampleOTT device 102 in the illustrated example. The example OTT device 102decodes received media and outputs the decoded media to the mediapresentation device 304 for presentation. The example media presentationdevice 304 may be a television, a monitor, an audio receiver, an audioamplifier, etc. In some examples, the example OTT device 102 isintegrated in the media presentation device 304.

In an example impression collection phase, like the one shown in FIG. 3, the example AME 106 collects impressions corresponding to mediaaccessed via the example OTT device 102. Examples disclosed hereincollect impression information for any type of media including contentand/or advertisements. Media may include advertising and/or content suchas web pages, streaming video, streaming audio, movies, and/or any othertype of content and/or advertisements delivered via satellite,broadcast, cable television, radio frequency (RF) terrestrial broadcast,Internet (e.g., internet protocol television (IPTV)), televisionbroadcasts, radio broadcasts and/or any other vehicle for deliveringmedia.

In the illustrated example, the example OTT device 102 employs mediastreaming applications (e.g., apps) and/or a web browser to accessmedia, some of which include instructions that cause the example OTTdevice 102 to report media monitoring information to the example AME106. That is, when the example OTT device 102 of the illustrated exampleaccesses media, an application and/or web browser of the example OTTdevice 102 executes instructions in the media to send an impressionrequest 306 to the example database proprietor 104 via, for example, theInternet or any other network. The impression request 306 of theillustrated example includes the example matched ID 208, a time stamp308, and a media ID 310 that identifies the media accessed at theexample OTT device 102 at the time indicated by the time stamp 308. Theexample time stamp 308 includes the date and time the media waspresented. In some examples, the time stamp 308 includes the durationthe media was presented. In some examples, the time stamp 308 includes astart time and a stop time.

The example media ID 310 may be a code, a signature, watermarkinformation, a content management system (CMS) tag, or any otheridentifying information that the example AME 106 pre-defines as beingassociated with particular media so that subsequently logged impressionsbased on the media ID 310 (such as the impressions logged during theimpression collection phase of FIG. 3 ) can be used to identify theparticular corresponding media. The example impression request 306allows the example database proprietor 104 to associate (e.g., link)impressions of different media accessed via the example OTT device 102with demographics corresponding to the example matched ID 208 storedwithin the example database proprietor 104 database, thus creating ademographic impression 312. For example, for a media presentation withina household, the demographics of the members of the household areassociated with a household impression of the media presentation.

In some examples, the example database proprietor 104 sends the exampledemographic impression 312 to the example AME 106 via, for example, theInternet or any other network. The example demographic impression 312 ofthe illustrated example includes the example matched ID 208, the mediaID 310, demographics 314, and the time stamp 308. In the illustratedexample, the example demographics 314 are the matched demographics 210associated with the example matched ID 208 determined in the exampleregistration phase illustrated in FIG. 2 . In the illustrated example,when the example AME 106 receives the example demographic impression 312from the example database proprietor 104, the example AME 106 logs anexample impression 316 in an example impression logs data structure 318based on the example matched ID 208. The example impression logs datastructure 318 includes the example matched ID 208 and the correspondingdemographics 314, which the example AME 106 obtains from the exampledatabase proprietor 104 in the example demographic impression 312.

In some examples, the example database proprietor 104 aggregatesdemographic impressions and sends the aggregate impressions to theexample AME 106. In other examples, the example database proprietor 104sends individual, user-level demographic impressions to the example AME106. In the illustrated example, the example demographic impressions 306are logged in the example impression logs data structure 318 toassociate the media ID 310 with the time stamp 308, the example matchedID 208, and the demographics 314. In this manner, the example AME 106can compare demographic impressions from the example database proprietor104 with demographic impressions from panelists, as disclosed herein.

FIG. 4 illustrates the example demographic corrector 110 of FIG. 1 . Theexample demographic corrector 110 utilizes demographics matched by theexample demographic comparator 107 (FIG. 1 ) along with AME panelistdemographic data stored in the example panelist database 116 (FIG. 1 ).In the illustrated example, the example demographic corrector 110corrects demographics from the example database proprietor 104 at ahousehold-level. For example, the example database proprietor 104 andthe example AME 106 provide demographics for particular households,rather than for every viewer (e.g., person-level data). In theillustrated example, a particular household is made up of numerousdemographic buckets (e.g., ages 2-12, 13-14, 15-17, 18-20, 21-24, 25-29,30-34, 35-39, 40-44, 45-49, 50-54, 55-64, 65+ for each gender).

The example demographic corrector 110 includes an example demographicimpression receiver 402, an example panelist database 404, an example DPdatabase 406, an example household manager 408, an example comparator410, an example analyzer 412, an example adjuster 414, an example modeltrainer 416, and an example model applier 418.

The example demographic impression receiver 402 of the exampledemographic corrector 110 receives demographic impressions from theexample demographic comparator 107. The example demographic impressionreceiver 402 stores demographic data associated with matched databaseproprietor demographic impressions within a DP database 406. Similarly,the example panelist database 404 receives and stores demographic dataand/or demographic impression data from the example panelist database116 (FIG. 1 ).

The example household manager 408 of the example demographic corrector110 obtains demographic data from the example panelist database 404and/or the example DP database 406. The example household manager 408determines a probability that a first demographic bucket exists in aparticular household. In the illustrated example, the example householdmanager 408 utilizes multiple household-level machine learning models topredict demographic groups (e.g., demographic buckets) actually existingin a particular household.

In the illustrated example, the example model trainer 416 trains anindependent binary model (e.g., yes or no) for each age and gendercombination corresponding to a demographic bucket (e.g., (male orfemale)×(ages 2-12, 13-14, 15-17, 18-20, 21-24, 25-29, 30-34, 35-39,40-44, 45-49, 50-54, 55-64, 65+)), totaling twenty-six binary models. Insome examples, short term visitors and members with ages less than twoyears old are excluded, and models are not trained for these particularpeople. While the present disclosure discusses twenty-six binary models,any number of models may be used without departing from the scope of thepresent disclosure. In the illustrated example, the example modeltrainer 416 utilizes a classification and regression tree (“CART”)technique to generate the example models. In some examples, the examplemodel trainer 416 utilizes a 5-folds cross validation to tune thecomplexity parameter of each CART model. Additionally or alternatively,the example model trainer 416 may use logit, conditional inference tree(“ctree”), random forest, neural networks, bootstrap aggregate decisiontrees, or other modeling techniques without departing from the scope ofthe present disclosure. The example model trainer 416 may utilizenumerous variables in building the example models, including, withoutlimitation, household (“HH”) size, age (mean age of the HH, number ofkids, number of teenagers, number of 18-24 year olds, number of 25-34year olds, number of people over 55, etc.), gender (number of female,male, etc.), person status (number of primary, number of dependents,etc.), income, education, ethnicity, etc. The example models produce aprobability that a particular demographic bucket exists in a particularhousehold. The example comparator 410 predicts whether the particulardemographic bucket exists in the particular household by comparing theprobability to a threshold. The threshold is selected to maximizemetrics of interest (e.g., sensitivity, accuracy, precision, etc.). Insome examples, the threshold is consistent across all demographicbuckets. In the illustrated example, each demographic bucket and thecorresponding independent binary model has a unique threshold (e.g., afirst threshold of 0.7 may be used for males between the ages of 18 and20 and a second threshold of 0.55 may be used for females between theages of 30-34).

In some examples, the existence of certain demographic buckets aredependent on the existence of other demographic buckets (e.g., childrendemographic buckets are likely dependent on adult demographic buckets).In such examples, a plurality of dependent binary models may be used toaccommodate such dependencies. In some examples, a two-stage model isused, such that dependencies are identified by a second set of binarymodels different from the above identified models.

In some examples, the example model trainer 416 trains an Ensemble ofClassifier Chains (“ECC”) model to incorporate demographic bucketdependencies in an iterative manner. In the ECC model, the example modeltrainer 416 trains m classifiers (e.g., C1, C2, C3, . . . , Cm) with arandom chain ordering (e.g., a chain arranging the twenty-sixdemographic buckets described herein) and a random subset of thetraining set of household data described herein. For example, theexample model trainer 416 iteratively adds the demographic bucket(s)determined to exist in the example household (e.g., by the examplecomparator 410) as a predictor variable to predict the next demographicbucket in the chain ordering for a classifier. Thereafter, the examplemodel trainer 416 aggregates, for each demographic bucket, the predicteddemographic buckets across the m chains. Each demographic bucketreceives m votes from the m chains. Thereafter, the example comparator410 compares the aggregate sum to a threshold to determine whether ornot the demographic bucket exists in the home. This ECC model mayproduce a different prediction value than the independent binary modelfor a particular demographic, thereby incorporating dependency variationinformation (e.g., adult demographics may be dependent on a childdemographics). Therefore, the household manager 408 can predict theexistence of particular demographic buckets more accurately whendependencies are accounted for, such as by the example process describedherein. Additionally or alternatively, the household manager 408 mayutilize multiple label classification methods (e.g., multi labelk-nearest-neighbor) to incorporate dependencies.

In the illustrated example the example analyzer 412 calculatesprobabilities for the number of individuals within the existingdemographic buckets for a particular household. The example analyzer 412determines conditional probabilities for the number of members (e.g.,one member, two members, three members, etc.) within the existingdemographic buckets based on demographic distributions of panelistswithin panelist households. For example, the conditional probability maybe the probability that there is one member in an existing demographicbucket based on the condition that there is at least one member in theexisting demographic bucket (e.g., one of the demographic buckets thatthe household manager 408 determined exists in the household). Theexample analyzer 412 defines intervals between zero and one based on theconditional probabilities. The number of internals may be dependent onthe number of outcomes (e.g., one person in an existing demographicbucket, two people, three people, four people, etc.). Of course, anynumber of intervals may be defined. The example analyzer 412 thengenerates a random number Rn between 0 and 1. The example comparator 410compares the generated random number Rn to the defined intervals andidentifies the number of people in the demographic bucket based on whichinterval the generated random number Rn falls into. Accordingly, theexample demographic corrector 110 predicts which demographics existwithin a household and predicts how many people are in each of theexisting demographic buckets.

In some examples, person-level matched demographic impressions areavailable or household-level matched demographic impressions areavailable and person-level correction is used. In such examples, thenumber of missing people within a household is predicted prior tocorrecting the demographics of the DP household members. In someexamples, a machine learning and/or statistical model (e.g., decisiontrees, neural networks, etc.) is used to predict the number of missingpeople as well as the demographics of the missing people associated witha household. In some examples, missing members can be determined usingmean and/or median values in similar HHs to predict the demographics ofthe missing members. Alternatively, a “missing” label is applied tovariables for which the database proprietor does not have data (e.g., nodata=“missing”).

Thereafter, the example demographic corrector 110 corrects demographicsfrom the example database proprietor 104. The example household manager408 prepares a panelist household member's age to be compared to DPhousehold members' ages where person-level matched data is notavailable. While the present disclosure discusses matching ages, anyother demographic information may be used similarly for finding matchesbetween AME panelists and database proprietor registrants withoutdeparting from the scope of this disclosure. In some examples, theexample household manager 408 matches a household first by getting bothpanelist and DP household member impression information associated withthe same matched ID.

In some examples, a machine learning and/or statistical model is used tocorrect individual level database proprietor demographics usingvariables, such as, for example, DP household size, DP demographicbuckets, person status (e.g., head of household, number of adult parentsin a household, number of unknown family members in a household),education (e.g., levels of education), age (e.g., number of kids ages0-12, number of teenagers ages 13-17, number of young adults ages 18-34,number of adults ages 35-45, number of elderly ages 65+), gender (e.g.,number of unknown gender, number of males, dominant gender), andethnicity (e.g., number of Caucasian, Hispanic, African American,dominant ethnicity).

In some examples, extra members can be determined based on age-gendercombinations (e.g., a male of age 33 would represent an ‘Extra’ memberwhere only females and/or males ages 1-27 are present). In otherexamples, extra members can be determined based on separate models forage and gender (e.g., model 1 for age buckets and model 2 for genderbuckets). In such examples, a machine learning and/or statistical modelis formed using the person-level matched data to predict for eachperson, whether that person is “extra” or not. In such examples,different “extra” members may be identified in each example. Therefore,in some examples, the accuracy of “extra” member identification isverified. In other examples, only members identified as “extra” membersare considered.

The example comparator 410 of the example demographic corrector 100compares a panelist to all DP household members for age comparison. Theexample analyzer 412 of the example demographic corrector 110 determinesthat the panelist and the DP household member with the closest ages area possible match. Thereafter, the example analyzer 412 verifies if thematching household members are within a variance of years (e.g., +3,+10, etc.). Examples disclosed herein utilize variances (e.g., +3, +10,etc.) to account for mistakes, lies, and/or other demographicinaccuracies that could lead to misattributions. Using variancesprovides for a larger number of matches to occur, however, it alsohighlights the fact that the demographic data is incorrect. Therefore,examples disclosed herein correct such errors in the demographic data.

While the example analyzer 412 determines that a panelist matches with aDP household member, the panelist and DP household member may havedifferent demographics (e.g., age, gender, etc.). In some examples,these differences can be adjusted (e.g., corrected) using the panelistdemographic information as truth. For example, if the DP memberdemographics vary from the panelist demographic information, then the DPmember demographics are incorrect and can be corrected by replacing theincorrect demographics with the panelist demographics. The exampleadjuster 414 of the example demographic corrector 110 corrects the DPhousehold member demographics by determining the difference(s) betweenthe demographic data of the DP household member and the panelist, andcorrecting the demographics, as discussed above. In other examples, theexample adjuster 414 only determines the difference(s) between thedemographic data of the DP household member and the panelist. Theexample model trainer 416 of the example demographic corrector 110trains a model based on the determined difference(s) in the demographicdata of the DP household member and the panelist. The example modelapplier 418 of the example demographic corrector 110 utilizes modelstrained by the example model trainer 416 to apply similar corrections toa plurality of demographic impressions, especially those where panelistdata is not available. Thus, based on a panelist subset (e.g., a samplepopulation), a statistical model is trained and applied to an entire set(e.g., a universe population). The statistical model may be a machine orcomputational learning algorithm (e.g., decision trees, neutral network,etc.). In some examples, the statistical model has the ability toperform accurately on new, unseen examples/tasks after havingexperienced a learning dataset.

In some examples, the example analyzer 412 cannot match a panelist to aDP household member (e.g., HH size in DP only matches AME HH size 20% ofthe time; 40% of households have missing members; 40% of households haveextra members). In the illustrated example, the example adjuster 414assigns a “missing” label or an “extra” label to the non-matched DPhousehold members, as will be further described below. In some examples,the demographics of missing and/or extra non-matched DP householdmembers are corrected after a model has been trained by the examplemodel trainer 416. Additionally or alternatively, the missing and/orextra non-matched DP household members may have demographics predictedby the example demographic predictor 114, because without a matchingpanelist it may be difficult to determine whether or not the DPhousehold member's demographics are accurate. In these examples, theexample demographic predictor 114 uses television and PC media andviewing history to predict demographics of a missing and/or extra DPhousehold member (e.g., children programming in an all adult householdmakes it probable there is child impression data missing from in thedatabase proprietor's database). In some examples, households withchildren are more likely to have missing and/or incorrect impressions.

In some examples, the number of missing people within a household ispredicted prior to correcting the demographics of the DP householdmembers. In some examples, there are no missing people in a household, 1missing person in a household, 2 missing people in a household, or morethan 2 missing people in a household. An example distribution is 51% ofhouseholds have no missing people, 24.5% of households have one missingperson, 12.5% of households have two missing people, and 12% ofhouseholds have more than two missing people. A number of factors affectdistributions such as the example distribution above. In some examples,DP household size, DP demographic buckets, person status (e.g., head ofhousehold, number of adult parents in a household, number of unknownfamily members in a household), education (e.g., levels of education),age (e.g., number of kids ages 0-12, number of teenagers ages 13-17,number of young adults ages 18-34, number of adults ages 35-45, numberof elderly ages 65+), gender (e.g., number of unknown gender, number ofmales, dominant gender), and ethnicity (e.g., number of Caucasian,Hispanic, African American, dominant ethnicity) factor into suchdistributions.

In some examples, extra members can be determined based on demographicbuckets having age-gender combinations (e.g., male age 33 wouldrepresent ‘Extra’ in demographic bucket of females and/or males ages1-27). In other examples, extra members can be determined based onseparate models for age and gender (e.g., a first model for age bucketsand a second model for gender buckets). Example flow diagrams of exampleoperations of the example demographic corrector 110 are described belowin connection with FIGS. 7, 8, and 9A-9C.

FIG. 5 illustrates the example viewership assigner 112 of FIG. 5 . Whencorrecting demographics as disclosed herein, the example demographicimpressions are adjusted and/or modeled based on demographics frompanelist data. In the illustrated example, viewership for a particularimpression is associated with the corrected demographics. As describedbelow in connection with FIG. 5 , the example viewership assigner 112utilizes the corrected demographics from the example demographiccorrector 110 to accurately assign viewership to the correct HH membersto create corrected demographic impressions. The example viewershipassigner 112 of the illustrated example of FIG. 5 includes an exampleOTT demographic impression receiver 502, an example OTT database 504, anexample donor data receiver 506, an example donor database 508, anexample household (“HH”) member manager 510, an example viewershipanalyzer 512, an example viewership adjuster 514, and an example OTTviewership assigner 516.

The example OTT demographic impression receiver 502 of the exampleviewership assigner 112 receives corrected demographics associated withthe corrected demographics from the example demographic corrector 110(FIG. 4 ), to assign viewership of a particular impression to aparticular HH member. Along with the corrected demographics, the exampleOTT demographic impression receiver 502 receives OTT device informationincluding impressions, times in which a household uses the example OTTdevice 102, and OTT media information (e.g., name, genre, applicationname, application category, time of day of exposure, etc.). Accordingly,the example OTT demographic impression receiver 502 sends the receivedimpressions, times, media information, and corrected demographics to anOTT database 504.

Additionally, the example donor data receiver 506 of the exampleviewership assigner 112 obtains media and viewing history of a panelistobtained through AME metering. In some examples, the example AME 106(FIG. 1 ) implements numerous methods to collect media impressioninformation (e.g., media, time, viewership, etc.) from televisionsand/or PCs. In some examples, metering applications are installeddirectly on a television, a set top box, and/or a PC to accuratelymeasure media and viewership directly. Other methods include monitoringmedia through proxies using proxy logs to determine media, time,viewership, etc. The example donor data receiver 506 receives media andviewing history information from the example AME 106 obtained throughthese or other alternative methods of collecting television and PC mediaand viewership. Accordingly, the example donor data receiver 506 sendsthe received times, media information, and viewing history informationto a donor database 508.

In the illustrated example, the example viewership assigner 112 utilizesprobabilities based on donor household data (e.g., panelist and/orhistorical viewing data), such as metering data associated withtelevisions connected to OTT devices or other metering as describedherein. In some examples, the example donor data receiver 506 acquiresdonor data via the example network 122 (block 1004). The example donordata receiver 506 stores the example donor data in the example donordatabase 508. The example viewership analyzer 512 uses a plurality ofvariables to find donor household information in the example donordatabase 508 that match with data related to the first household.Examples of matching variables include, without limitation, demographics(male ages 2-12, 13-4, 15-17, etc., female ages 40-44, 50-54, etc.),time of day (e.g., prime time, late afternoon/early evening, lateevening, weekday morning, weekday daytime, weekend daytime, etc.),specific genre (e.g., musical drama, feature film, comedy variety,children, etc.), HH size, number of adults, number of kids, etc. Thesematching variables provide, for example, who was viewing what was beingviewed, when was it being viewed, and how much was viewed. For example,over a period of six months, televisions connected to OTT devices mayprovide the program, originator, household identifier, viewing date,time of day, and demographics. In such examples, 930,420 viewing events(e.g., 22 million minutes) from 2,933 panelist spread across 869 homesmay provide 12,937 programs of various genres. In some examples, directmetering methods (e.g., people meter, portable people meter, etc.)collect such information. In some examples, attribution viabutton-pushing provides such information. In some examples, the exampleviewership adjuster 514 rolls up the variables (e.g., subsequentlysearches for matching donor household information with less variablesused in the search), such that a match may be found. In some examples,initially a donor household must match all matching variables associatedwith the first household. After the example viewership adjuster 514rolls up the variables a first time, a donor household must match lessvariables than before.

The example viewership analyzer 512 accesses the actual demographicbuckets that exist within the first household based on the demographicscorrected by the example demographic corrector 110. For a firstdemographic bucket, the example viewership analyzer 512 determines aprobability that a viewer in the first household viewed a mediapresentation by dividing a first time that a person in the donorhousehold matching the first demographic bucket previously viewed themedia presentation (e.g., based on panelist viewing history) by a secondtime that all members in the donor household previously viewed the mediapresentation. For example, an OTT impression was recorded for the firsthousehold on Thursday at 8 pm during The Big Bang Theory. The firsthousehold has three people with demographics corrected by thedemographic corrector: Linda (F45), Mike (M44), and Eliza (F13). Theexample viewership assigner 112 determines the probability that Lindawas watching the OTT device at the time of the impression by identifyingdonor data including viewing events associated with the demographicfemale ages 45-49, during primetime, with a situational comedy viewinggenre in a household size of three in a home that has <=two adults andone child. In such examples, the example viewership assigner 112identifies all viewing events for households with 3 people, <=2 adults,1 child, and at least one person in the F45-49 demo for situationcomedies during primetime. Within this subset, the example viewershipassigner 112 calculates the probability of Linda viewing The Big BangTheory by adding up the total minutes viewed (e.g., by someone in theF45-49 demographic bucket) and dividing by the total possible minutesviewed (e.g., all minutes that the program was on in the subset ofhouseholds). The example viewership assigner 112 repeats this procedurefor Mike and Eliza. Of course, the probability determinations may becalculated and stored in look-up tables for the first household (e.g.,in the example OTT database 504) and for the donor households (e.g., inthe example donor database 508) such that the probabilities need not bere-calculated over and over.

Thereafter, the example viewership analyzer 512 searches for a donorviewing event matching, for example, the time of day and genre of animpression, and the first household size, number of adults and number ofchildren in the first household. If there is more than one possiblematch, the example viewership analyzer 512 ranks the first householdmembers and the household members from all donor viewing events fromhighest probability of viewing the media presentation to lowest. In someexamples, such as with households of five or more people, the number ofdonor viewing events is low as there are not many households with morethan five members. In such examples, only the five members with thehighest probabilities of viewing the media presentation (of both thedonor and recipient homes) are used for matching. For each firsthousehold member, the example viewership analyzer 512 subtracts theprobability that the household member viewed the media presentation fromthe probability that the corresponding donor household member viewed themedia presentation, taking the absolute value of the difference. Theexample viewership analyzer 512 adds up the differences in probabilitiesacross all household members. This process is repeated for all possibledonor households/viewing events, and the example viewership analyzer 512selects the donor viewing event with the smallest difference as thedonor. In the case of a tie, the donor viewing event is chosen randomly.

When the example viewership analyzer 512 identifies a donor viewingevent, the first household members are matched to donor householdmembers with the same rank (e.g., ranked from highest probability ofviewing the media presentation to lowest). For example, the firsthousehold member with the highest probability of viewing the mediapresentation is matched to the donor household member with the highestprobability of viewing the media presentation. Thereafter, the viewinghistory (e.g., viewed or did not view the media presentation) from thematched donor household members is imputed onto the members of the firsthousehold, thereby assigning viewership for a particular impression ofthe media presentation. Accordingly, the example viewership analyzer 512identifies instances of co-viewing (e.g., where more than one person iswatching the same media presentation), thereby increasing the number ofimpressions. For example, where there is one impression per household,but two people are viewing, the example viewership analyzer 512 assignsthe impression to two people.

Additionally or alternatively, the example HH member manager 510 of theexample viewership assigner 112 utilizes the example OTT deviceimpressions, times, media information, and corrected demographics fromthe example OTT database 504 and the television and PC times, mediainformation, and viewing history information from the example donordatabase 508 to predict which HH member is using the example OTT devicefor each particular impression. The example HH member manager 510historically extracts media and viewing history from the example donordatabase 508 to predict and/or match a HH member to a particularimpression. In other words, media and viewing history from televisionand PC is a factor in determining OTT device viewership for similarmedia and/or during similar times in some examples.

The example HH member manager 510 also distinguishes particular OTTmedia. In some examples, media impressions that have media informationassociated therewith (e.g., from the example OTT database 504) aredetermined to correspond to media accessed using an application (e.g.,advertisements embedded in video) running on OTT devices. In otherexamples, media impressions that have no media information associatedtherewith (e.g., absent in the example OTT database 504) are determinedto correspond to media accessed via a webpage, homepage, channel, orother non-application media.

In some examples, multiple household members are to be assigned to aparticular impression (e.g., multiple people are watching the same mediaat the same time). The example viewership analyzer 512 of the exampleviewership assigner 112 determines whether multiple HH members are to beassigned viewership in association with a single impression. In someexamples, a backward elimination model is used to determine viewership(e.g., all HH members are accounted for and those HH members that aredetermined not to be viewers are eliminated). In some examples, aviewership adjuster 514 removes HH members that are determined not to beviewers. In some examples, the example viewership adjuster 514 adds HHmembers that are determined to be viewers.

In some examples, the example viewership analyzer 512 analyzes viewingand household information for a particular household (recipient) anduses panelist viewing (donor) (e.g., OTT media, television, and PC mediaviewing history) to assign probabilities of viewing to all householdmembers. In some examples, the example viewership analyzer 512determines a set of donor households based on the recipients' viewingand household characteristics to assign probabilities of viewing to alldonor household members. In some examples, the example viewershipanalyzer 512 compares recipient households to donor households to findmatches. In some examples, the matches are compared to a thresholdprobability. In some examples, viewership is assigned to recipienthouseholds based on the assigned viewership of a matched donorhousehold.

In some examples, the example viewership analyzer 512 uses mediainformation (e.g., program, time, etc.) and both household andindividual characteristics to assign probabilities of viewing to eachhousehold member. In some examples, the example viewership analyzer 512creates a statistical and/or machine learning model based on panelistviewing history and panelist household and individual characteristics.The example viewership analyzer 512 applies the model to a non-panelisthousehold to assign viewership according to the probabilities of viewingand/or a pre-determined threshold.

The example OTT viewership assigner 510 of the example viewershipassigner 112 associates an impression to the HH member(s) (and his orher demographic information) that are determined to be viewers by theexample viewership adjuster 514 and the example viewership analyzer 512.In some examples, panelist viewership based on television and/or PCmedia and timestamps is scaled and similarly assigned to OTT deviceviewership. In other examples, the example OTT viewership assigner 510does not account for co-viewing and a single impression is associatedwith a corresponding single HH member. Other examples for accounting forco-viewing may be utilized without departing from the scope of thepresent disclosure. An example flow diagram illustrating an exampleimplementation of the example viewership assigner 112 is described belowin connection with FIG. 9 .

FIG. 6 illustrates an example demographic predictor of FIG. 1 to predictdemographics for impressions not corrected by the example demographiccorrector. In some examples, the example demographic comparator 107(FIG. 2 ) does not match user data 204 (FIG. 2 ) from the example OTTdevice 102 (FIG. 2 ) to demographics 206 (FIG. 2 ) from the exampledatabase proprietor 104 (FIG. 2 ), and/or the example demographiccorrector 110 (FIG. 4 ) does not match a panelist HH member to a DPhousehold member, leaving a plurality of unknown demographicimpressions. Therefore, to obtain corrected demographic impressions inthese examples, the example demographic predictor 114 is utilized. Inthe illustrated example of FIG. 6 , the example demographic predictor114 predicts demographics for OTT-DP impressions not matched by theexample demographic comparator 107.

The example demographic predictor 114 of the illustrated example of FIG.6 includes an example OTT demographic impression receiver 602, anexample OTT-DP matched impression database 604, an example OTT-DPnon-matched impression database 606, an example IP address demographicreceiver 608, an example prediction manager 610, an example linearscaler 612, an example differential scaler 614, and an example modeler616.

In the illustrated example, to predict demographics for a particularimpression, the example demographic predictor 114 receives the matchedand non-matched demographic impressions from the example demographiccomparator 107. Accordingly, the example demographic predictor 114stores the matched demographic impressions in the example OTT-DP examplematched impressions database 604. Similarly, the example demographicpredictor 114 stores the non-matched demographic impressions in theexample OTT-DP non-matched impressions database 606.

In the illustrated example, the prediction manager 610 utilizes thematched impressions from the example OTT-DP matched impressions database604 to create predictions for non-matched demographic impressions fromthe example OTT-DP non-matched impressions database 606. In someexamples, the demographics from the matched impressions are scaled tomake up for missing demographics in the non-matched impressions.

In the illustrated example, the example demographic predictor 114includes the linear scaler 612 to linearly scale demographics from thematched impressions (e.g., impressions from the example OTT device 102matched with data from the example database proprietor 104, such as, forexample, panelist data). The difference between the scaled demographicsand the demographics from the matched impressions can be used torepresent the demographics for the non-matched impressions. For example,if 95% of all the impressions are matched and 5% of all the impressionsare not matched, the linear scaler 612 can scale the demographics acrossall demographic buckets from 95% to 100% (e.g., a demographicdistribution of the demographics from the matched impressions can bemultiplied by 100/95). For example, if the demographic distribution fromthe above example includes 95 twenty-five year old males associated withthe matched impressions, 95 can be multiplied by 100/95 such that thedemographic distribution now includes 100 twenty-five year old males(and the additional 5 twenty-five year old males can be used for thenon-matched impressions). In such examples, it is presumed that the 5%of the impressions that are not matched are similar to the 95% of theimpressions that are matched. The example linear scaler 612 applies aconstant scaling factor to all demographic buckets.

In some examples, the constant scaling factor may be based onco-viewing, as determined by the example viewership assigner 112 (e.g.,directly obtained from the example viewership assigner 112, receivedfrom the example viewership assigner 112 over the example network 122,etc.). For example, the example viewership assigner 112 accounts forco-viewing by identifying multiple household members associated with animpression, thereby counting multiple impressions from a singlehousehold impression. Accordingly, the number of impressions determinedby the example viewership assigner 112 may be larger than the number ofimpressions the example OTT demographic impression receiver 502 (FIG. 5) initially receives. The example linear scaler 612 and/or the exampledifferential scaler 614 generates a co-viewing rate (“CVR”) based on thenumber of demographic impressions determined by the example viewershipassigner 112 and the number of matched demographic impressions receivedby the example OTT demographic impression receiver 502 (e.g., theexample viewership assigner 112 determines there are 1200 impressionswhen the example OTT demographic impression receiver 502 received 950(e.g., out of 1000, such that 50 were not matched by the exampledemographic comparator 107 (FIG. 1 )) such that the co-viewing rate maybe CVR=1200/950=1.263). The example linear scaler 612 may scale thenon-matched impressions (e.g., 50) based on the co-viewing rate (e.g.,1.263) to account for co-viewing (e.g., 63 impressions). In suchexamples, the example modeler 616 distributes the co-viewing scaledimpressions across the various demographic buckets. Thereafter, theexample linear scaler 612 combines the scaled up non-matched impressions(e.g., 63) and the number of impressions determined by the exampleviewership assigner 112 (e.g., 1263) and divides by the number ofimpressions determined by the example viewership assigner 112 (e.g.,1200) to obtain the constant scale factor.

In some examples, a unique audience value may be scaled by the examplescaling factor. The unique audience value defines a distinct number ofpeople who viewed a media presentation within a given period of time(e.g., identified in panelist data, calculated and/or adjusted afterassigning viewership, etc.). The unique audience value may be differentfrom the number of impressions as a unique individual may view the samemedia presentation multiple times (e.g., there may be a total of 500impressions but only 255 unique people in a particular demographicbucket). The unique audience value can be scaled to account forco-viewing similarly to the non-matched impressions discussed above. Theexample linear scaler 612 may divide the number of impressionsdetermined by the example viewership assigner 112 (e.g., 1200) by theunique audience value (e.g., 630) to determine a frequency (e.g., 1.90).Thereafter, the example linear scaler 612 may divide the combined scaledup non-matched impressions and number of impressions determined by theexample viewership assigner 112 (e.g., 1263) by the frequency (e.g.,1.90) to determine a scaled unique audience value (e.g., 664). Ofcourse, the non-matched impressions and the unique audience value may bescaled across the various demographic buckets (e.g., male ages 2-12,female ages 2-12, etc.) and/or across the aggregate of the demographicbuckets.

In other examples in which the non-matched impressions cannot beaccurately scaled linearly, the example demographic predictor 114includes the example differential scaler 614 to differentially scaledemographics from the matched impressions. For example, instead ofapplying a constant scaling factor to all demographic buckets, theexample differential scaler 614 applies a variable scaling factor toeach demographic bucket individually. In the illustrated example, thevariable scaling factor is based on a match rate of panelists for aparticular demographic bucket. For example, if 95% of the male panelistsof ages 15-17 are matched by the example demographic comparator 107(FIG. 1 ) (e.g., 5% of the male panelists of ages 15-17 are notmatched), then the example differential scaler 614 applies a firstscaling factor (e.g., 100/95) to the demographic bucket associated withmales of ages 15-17. Similarly, if 80% of the female panelists of ages40-44 are matched by the example demographic comparator 107 (e.g., 20%of the female panelists of ages 40-44 are not matched), then the exampledifferential scaler 614 applies a second scaling factor (e.g., 100/80)to the demographic bucket associated with females of ages 40-44. Asdiscussed above, the difference between the scaled demographics and thedemographics from the matched impressions can be used to represent thedemographics for the non-matched impressions. Thus, the panelist datamatch rate is associated with the scaling factor for each demographicbucket.

Additionally or alternatively, the prediction manager 610 receives thecorrected demographics from the example demographic corrector 110 (FIG.4 ) with the example OTT demographic impression receiver 602. In theillustrated example, the prediction manager 610 utilizes the correcteddemographics from the example demographic corrector 110 to createpredictions for unknown demographic impressions. In some examples, thecorrected demographics are scaled to allow demographic associations tobe made with the non-matched impressions. The corrected demographics maybe scaled linearly or differentially as disclosed herein.

In some examples, demographic impressions can be matched to particularusers via IP addresses. In some examples, an additional or alternatedatabase proprietor (e.g., eXelate) provides demographics for aparticular user based upon the user's IP address. In some examples, IPaddress-based demographic matching occurs when the example demographiccomparator 107 cannot match the first set of data to the second set ofdata. In other examples, IP address-based demographic matching occurs inparallel or in replacement of the example demographic comparator 107. Inthe illustrated example, the example demographic predictor 114 includesthe example IP address demographic receiver 608 to receive demographicdata for a particular user based on the user's IP address. Thereafter,the example prediction manager 610 matches the non-matched impressionsfrom the example OTT-DP non-matched impressions database 606 with theuser demographic data based on the corresponding IP address of theuser's OTT device. Additionally or alternatively, the example predictionmanager 610 uses the IP address of a household modem, router,local-area-network, or other private network.

In the illustrated example, the example demographic predictor 114includes the example modeler 616 to train a prediction model based onthe matched impression scaling, the corrected demographic scaling,and/or the IP address demographic matching described above. In someexamples, the example modeler 616 applies the prediction model to theexample OTT-DP matched impressions from the example OTT-DP matchedimpressions database 604, the example OTT-DP non-matched impressionsfrom the example OTT-DP non-matched impressions database 606, and/or thecorrected OTT demographics from the example demographic corrector 110.In the illustrated example, the modeler 616 trains the prediction modelusing an AME panelist subset, and the prediction model is applied toaggregate demographic impressions. The aggregate demographic impressionsmay or may not include the AME panelist subset. In this manner, theexample demographic predictor 114 predicts the demographics fornon-matched impressions.

In examples disclosed herein, after the demographics have been correctedby the example demographic corrector 110 (FIG. 4 ), the viewership hasbeen assigned by the example viewership assigner 112 (FIG. 5 ), and theunknown impressions are predicted by the example demographic predictor114 (FIG. 6 ), the results are aggregated together by an exampleaggregator 118 (FIG. 1 ). Subsequently, the example apparatus 108 (FIG.1 ) generates and/or exports a plurality of reports 120 with theassigned viewership and corrected/predicted demographics, to reportdemographic impressions with relatively higher accuracies from OTTdevices.

While an example manner of implementing the example demographiccorrector 110 of FIG. 1 is illustrated in FIG. 4 , an example manner ofimplementing the example viewership assigner 112 of FIG. 1 isillustrated in FIG. 5 , and an example manner of implementing theexample demographic predictor 114 of FIG. 1 is illustrated in FIG. 6 ,one or more of the elements, processes and/or devices illustrated inFIGS. 4, 5, and 6 may be combined, divided, re-arranged, omitted,eliminated and/or implemented in any other way. Further, the exampledemographic impression receiver 402, the example panelist database 404,the example DP database 406, the example household manager 408, theexample comparator 410, the example analyzer 412, the example adjuster414, the example model trainer 416, the example model applier 418,and/or, more generally, the example demographic corrector 110 of FIG. 4; the example OTT demographic impression receiver 502, the example OTTdatabase 504, the example donor database 508, the example householdmember manager 510, the example OTT viewership assigner 510, the exampleviewership analyzer 512, the example viewership adjuster 514, theexample donor data receiver 506, and/or, more generally, the exampleviewership assigner 112 of FIG. 5 ; and/or the example OTT demographicimpression receiver 602, the example OTT-DP matched impression database604, the example OTT-DP non-matched impression database 606, the exampleprediction manager 610, the example linear scaler 612, the exampledifferential scaler 614, the example modeler 616, the example IP addressdemographic receiver 608, and/or, more generally, the exampledemographic predictor 114 of FIG. 6 may be implemented by hardware,software, firmware and/or any combination of hardware, software and/orfirmware. Thus, for example, any of the example demographic impressionreceiver 402, the example panelist database 404, the example DP database406, the example household manager 408, the example comparator 410, theexample analyzer 412, the example adjuster 414, the example modeltrainer 416, the example model applier 418, and/or, more generally, theexample demographic corrector 110 of FIG. 4 ; the example OTTdemographic impression receiver 502, the example OTT database 504, theexample donor database 508, the example household member manager 510,the example OTT viewership assigner 510, the example viewership analyzer512, the example viewership adjuster 514, the example donor datareceiver 506, and/or, more generally, the example viewership assigner112 of FIG. 5 ; and/or the example OTT demographic impression receiver602, the example OTT-DP matched impression database 604, the exampleOTT-DP non-matched impression database 606, the example predictionmanager 610, the example linear scaler 612, the example differentialscaler 614, the example modeler 616, the example IP address demographicreceiver 608, and/or, more generally, the example demographic predictor114 of FIG. 6 could be implemented by one or more analog or digitalcircuit(s), logic circuits, programmable processor(s), applicationspecific integrated circuit(s) (ASIC(s)), programmable logic device(s)(PLD(s)) and/or field programmable logic device(s) (FPLD(s)). Whenreading any of the apparatus or system claims of this patent to cover apurely software and/or firmware implementation, at least one of theexample demographic impression receiver 402, the example panelistdatabase 404, the example DP database 406, the example household manager408, the example comparator 410, the example analyzer 412, the exampleadjuster 414, the example model trainer 416, the example model applier418, and/or, more generally, the example demographic corrector 110 ofFIG. 4 ; the example OTT demographic impression receiver 502, theexample OTT database 504, the example donor database 508, the examplehousehold member manager 510, the example OTT viewership assigner 510,the example viewership analyzer 512, the example viewership adjuster514, the example donor data receiver 506, and/or, more generally, theexample viewership assigner 112 of FIG. 5 ; and/or the example OTTdemographic impression receiver 602, the example OTT-DP matchedimpression database 604, the example OTT-DP non-matched impressiondatabase 606, the example prediction manager 610, the example linearscaler 612, the example differential scaler 614, the example modeler616, the example IP address demographic receiver 608, and/or, moregenerally, the example demographic predictor 114 of FIG. 6 is/are herebyexpressly defined to include a tangible computer readable storage deviceor storage disk such as a memory, a digital versatile disk (DVD), acompact disk (CD), a Blu-ray disk, etc. storing the software and/orfirmware. Further still, the example demographic corrector 110 of FIG. 4, the example viewership assigner 112 of FIG. 5 , and/or the exampledemographic predictor 114 of FIG. 6 may include one or more elements,processes and/or devices in addition to, or instead of, thoseillustrated in FIGS. 4, 5 , and/or 6, and/or may include more than oneof any or all of the illustrated elements, processes and devices.

Flowcharts representative of example machine readable instructions forimplementing the example demographic corrector 110 of FIGS. 1 and 4 tocorrect demographics associated with OTT-DP matched impressions areshown in FIGS. 7, 8, and 9A-9C. Flowcharts representative of examplemachine readable instructions for implementing the example viewershipassigner 112 of FIGS. 1 and 5 to assign viewership to a correspondingsingle impression based on the corrected demographics from the exampledemographic corrector 110 are shown in FIGS. 10-11 . A flowchartrepresentative of example machine readable instructions for implementingthe example demographic predictor 114 of FIGS. 1 and 6 to predictdemographics for the example OTT-DP impressions that were not matched isshown in FIG. 12 . In these examples, the machine readable instructionscomprise programs for execution by a processor such as the processor1112 shown in the example processor platform 1100 discussed below inconnection with FIG. 13 . The programs may be embodied in softwarestored on a tangible computer readable storage medium such as a CD-ROM,a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-raydisk, or a memory associated with the processor 1112, but the entireprograms and/or parts thereof could alternatively be executed by adevice other than the processor 1112 and/or embodied in firmware ordedicated hardware. Further, although the example programs are describedwith reference to the flowcharts illustrated in FIGS. 7, 8A-8C, 9, and10 , many other methods of implementing the example demographiccorrector 110, the example viewership assigner 112, and/or the exampledemographic predictor 114 may alternatively be used. For example, theorder of execution of the blocks may be changed, and/or some of theblocks described may be changed, eliminated, or combined.

As mentioned above, the example processes of FIGS. 7, 8, 9A-9C, 10, 11,and 12 may be implemented using coded instructions (e.g., computerand/or machine readable instructions) stored on a tangible computerreadable storage medium such as a hard disk drive, a flash memory, aread-only memory (ROM), a compact disk (CD), a digital versatile disk(DVD), a cache, a random-access memory (RAM) and/or any other storagedevice or storage disk in which information is stored for any duration(e.g., for extended time periods, permanently, for brief instances, fortemporarily buffering, and/or for caching of the information). As usedherein, the term tangible computer readable storage medium is expresslydefined to include any type of computer readable storage device and/orstorage disk and to exclude propagating signals and transmission media.As used herein, “tangible computer readable storage medium” and“tangible machine readable storage medium” are used interchangeably.Additionally or alternatively, the example processes of FIGS. 7, 8,9A-9C, 10, 11, and 12 may be implemented using coded instructions (e.g.,computer and/or machine readable instructions) stored on anon-transitory computer and/or machine readable medium such as a harddisk drive, a flash memory, a read-only memory, a compact disk, adigital versatile disk, a cache, a random-access memory and/or any otherstorage device or storage disk in which information is stored for anyduration (e.g., for extended time periods, permanently, for briefinstances, for temporarily buffering, and/or for caching of theinformation). As used herein, the term non-transitory computer readablemedium is expressly defined to include any type of computer readablestorage device and/or storage disk and to exclude propagating signalsand transmission media. As used herein, when the phrase “at least” isused as the transition term in a preamble of a claim, it is open-endedin the same manner as the term “comprising” is open ended.

FIG. 7 is a flow diagram representative of example machine readableinstructions that may be executed to implement the example demographiccorrector 110 of FIGS. 1 and 4 . The example program 700 begins at block702 at which the example demographic impression receiver 402 receiveshousehold-level matched OTT-DP impressions matched by the exampledemographic comparator 107 (FIG. 1 ). The example demographic impressionreceiver 402 stores demographic data associated with each household inthe example DP database 406. The example panelist database 404 receivespanelist demographic data associated with panelist households from theexample panelist database 116 (FIG. 1 ).

From the example demographic data of household stored in the example DPdatabase 406, the example household manager 408 selects a household(block 704). The example demographic corrector 110 trains a model usinga training set of household data from a number of households.Thereafter, the example demographic corrector 110 applies the trainedmodel to all the households to correct demographics stored by theexample database proprietor.

Once a household has been selected, the example household manager 408identifies a demographic bucket to which a household member associatedwith the demographic data from the example database proprietor 104belongs (block 706). For example, from the example demographic data inthe example DP database 406, the example household manager 408 selects afirst demographic bucket (e.g., male ages 2-12) to determine whether aparticular member from the selected household is associated with thefirst demographic bucket. As discussed above, each demographic buckethas an independent binary model associated therewith.

The example household manager 408 identifies the probability that thefirst demographic bucket exists (e.g., there is at least one householdmember of that demographic) in the particular household based on theexample model corresponding to the first demographic bucket (e.g., 80%likelihood the first demographic bucket exists in the selectedhousehold). In some examples, the models corresponding to eachdemographic bucket are trained using panelist data to minimize thedifferences between panelist data (e.g., truth) and the probabilitiesthat the demographic buckets exist in a particular household. In suchexamples, the models are trained until the differences are minimized.Thereafter, subsequent demographic data corresponding to non-panelistsmay be adjusted, predicted, or otherwise corrected using these models.As described herein, machine learning algorithms, such as, for example,CART, logit, ctree, random forest, neural networks, or bootstrapaggregate decision trees are used to determine the probabilities that ademographic bucket is in a household. Other known methods of determiningthe probabilities that a demographic bucket is in a household may beused.

To determine whether a particular demographic exists in a particularhousehold, the example comparator 410 determines whether the probabilitythat the first demographic bucket is within the household satisfies afirst threshold (e.g., 67%) (block 708). If the probability that thefirst demographic bucket is within the household is not above the firstthreshold (block 708: NO), then control proceeds to block 710. At block710, the example household manager 408 determines if there areadditional demographic buckets to test. If there are additionaldemographic buckets to test (block 710: YES), control returns to block706. In some examples, the first threshold is consistent across alldemographic buckets. In the illustrated example, each demographic buckethas a unique threshold (e.g., the first threshold is 0.67 for malesbetween the ages of 18 and 20 and the first threshold is 0.6 for femalesbetween the ages of 30-34). Accordingly, the first threshold may varybased on the demographic bucket. In some examples, the first thresholdis based on a probability distribution of panelists within panelisthouseholds. For example, there may be a female aged between 30 and 34 in60% of panelist households, so the first threshold for females agedbetween 30 and 34 may be set to 0.6. Of course, the first threshold maybe determined using other known methods, such as, an output of a machinelearning algorithm, based on statistical analysis, etc.

If the probability that the first demographic bucket is within thehousehold is above the first threshold (block 708: YES), then controlproceeds to block 712. At block 712, the example analyzer 412 calculatesprobabilities for the number of individuals within the first demographicbucket. The example analyzer 412 determines conditional probabilitiesfor the number of members (e.g., one member, two members, three members,etc.) within the first demographic bucket based on demographicdistributions of panelists within panelist households. For example, theconditional probability may be the probability that there is one memberin the first demographic bucket on the condition that there is at leastone member in the first demographic bucket (e.g., that the firstdemographic bucket is in the household (block 708)). In such examples,the probability that there is one person in the first demographic bucketmay be 0.70, the probability that there are two people in the firstdemographic bucket may be 0.15, the probability that there are threepeople in the first demographic bucket may be 0.15. Of course, variousother probability distributions may be used as the distribution may bebased on the outcomes (e.g., number of people in the demographic bucket)and the probability of the occurrences of the outcomes.

In some examples, the example model trainer 416 train models usingmachine learning algorithms to determine the conditional probabilitiesfor the number of members (e.g., one member, two members, three members,etc.) within the demographic buckets. As discussed herein, the modelsmay be trained using based panelist information within panelisthouseholds. In such examples, the models are trained using the panelistinformation as truth to minimize the difference between the calculatedprobabilities for the number of members in a demographic bucket and thenumber of panelists in demographic buckets in panelist homes. Theexample models are trained until such differences are minimized.Thereafter, subsequent demographic data corresponding to non-panelistsmay be adjusted, predicted, or otherwise corrected using these models.As described herein, machine learning algorithms, such as, for example,CART, logit, ctree, random forest, neural networks, or bootstrapaggregate decision trees are used to determine the conditionalprobabilities of the number of members within each demographic bucket(e.g., determined via block 708). Other known methods of determining theprobabilities that a demographic bucket is in a household may be used.

In some examples, the process of block 712 is dependent on the processof block 708. In some examples, the processes of blocks 708 and blocks712 may be combined to determine the number of people in demographicbuckets that exist within a household in a single step. In suchexamples, the example model trainer 416 creates a single model using amachine learning algorithm such as, for example, CART, logit, ctree,random forest, neural networks, or bootstrap aggregate decision trees,etc., to directly predict which demographic buckets exist within ahousehold and how many members are in each demographic bucket.

In the illustrated example, the example analyzer 412 defines intervalsbetween zero and one based on the probabilities calculated at block 712(block 713). For example, the example analyzer 412 defines a firstinterval from 0 to 0.69 corresponding to the probability that there isone person in the first demographic bucket (e.g., 0.7), defines a secondinterval from 0.7 to 0.84 corresponding to the probability that thereare two people in the first demographic bucket (e.g., 0.15), and definesa third interval from 0.85 to 1 corresponding to the probability thatthere are three people in the first demographic bucket (e.g., 0.15). Ofcourse, any number of intervals may be defined to accommodate the numberof outcomes (e.g., one person, two people, three people, four people,etc.). The example analyzer 412 generates a random number Rn between 0and 1 (block 714). The example comparator 410 compares the generatedrandom number Rn to the defined intervals and identifies the number ofpeople in the demographic bucket based on which interval the generatedrandom number Rn falls into (block 716). For example, if 0≤Rn≤0.69, theexample comparator 410 identifies that there is one person in the firstdemographic bucket, if 0.7≤Rn≤0.84, the example comparator 410identifies that there are two people in the first demographic bucket,and if 0.85≤Rn≤1, the example comparator 410 identifies that there arethree people in the first demographic bucket. Thereafter, controlreturns to block 710.

Alternatively, the example comparator 410 may determine, for each numberof members (e.g., one, two, three, etc.), whether the probability thatthe number of members are in the first demographic bucket satisfies asecond threshold (e.g., 75%). For example, when the probability thatthere is one person in the first demographic bucket is 0.80 and theprobability that there are two people in the first demographic bucket is0.15, only the probability that there is one person in the firstdemographic bucket satisfies the second threshold of 0.75. Therefore, ifthe probability that the number of members are in the first demographicbucket is above the second threshold, then the example analyzer 412determines that that number of members (e.g., one) is within the firstdemographic bucket (block 716) and control returns to block 710. Ifthere are multiple numbers of members having probabilities satisfyingthe second threshold (e.g., second threshold is 30%, the probability oftwo members in the first demographic bucket is 35% and the probabilityof one member in the first demographic bucket is 40%), the secondthreshold may be adjusted (e.g., increased to 36%). Additionally oralternatively, the example analyzer 412 identifies the number of membershaving the highest probability (e.g., 40% probability that one member isin the first demographic bucket) as being in the first demographicbucket. For any number of members having a probability of being in thefirst demographic bucket lower than the second threshold, these numberof members are excluded from being in the first demographic bucket andcontrol returns to block 710. In some examples, the second threshold isthe same across all demographic buckets. In the illustrated example,each demographic bucket has a unique threshold (e.g., the secondthreshold is 0.75 for males between the ages of 18 and 20 and the secondthreshold is 0.7 for females between the ages of 30-34).

At block 710, if there are no additional demographic buckets to test(block 710: NO), the control proceeds to block 718. In some examples,the example model trainer may utilize a machine learning algorithm tocreate a corrective model to adjust the demographic data received fromthe demographic comparator 107. In such examples, the example adjuster414 adjusts the demographic data received from the example databaseproprietor 104 based on the demographic buckets identified in block 708and the number of members in each demographic bucket identified in block716 (block 718). The example model trainer 416 trains the correctivemodel based on the adjustments made in block 718 (block 720). If asufficient amount of households have not been used to train the examplemodel (block 722: NO), control returns to block 704. The sufficientamount of households may be determined to minimize the differencesbetween the corrected demographics and a panelist distribution, forexample.

This process may continue for a number of training iterations until theexample model performs accurately on unseen data (e.g., machine learningprocess). The example model trainer 416 may utilize any machine learningalgorithm, such as, for example, CART, logit, ctree, random forest,neural networks, or bootstrap aggregate decision trees, etc. After themodel has been trained by a sufficient amount of households (block 722:YES), the example model applier 418 applies the trained model to theremaining demographic data from the example database proprietor 104matched by the example demographic comparator 107 (block 724). In someexamples, the example model applier 418 applies the trained model to newdata received after the model(s) have been trained. For example, oncethe models described above have been trained (e.g., perform accuratelyon unseen tasks), the example model applier 418 can apply new data tothe models to correct the same. Additionally or alternatively, theexample models may be adjusted using the techniques described hereinafter being trained to increase the accuracy of the models as new datais provided. Thereafter, example program 700 ceases.

FIG. 8 is another flow diagram representative of example machinereadable instructions that may be executed to implement the exampledemographic corrector 110 of FIGS. 1 and 4 . The example program 800begins at block 802 at which the example OTT demographic impressionreceiver 402 (FIG. 4 ) receives or otherwise obtains aggregatedemographic impressions from the example database proprietor 104 (FIG. 1) (block 802). The example OTT demographic impression receiver 402further receives panelist demographic impressions from the examplepanelist database 116 (FIG. 1 ) (block 804). For example, the exampledemographic impression receiver 402 receives matched OTT-DP impressionsmatched by the example demographic comparator 107 (FIG. 1 ). In someexamples, aggregate demographic impressions are unavailable. In suchexamples, demographic impressions are created with the exampleviewership assigner 114, as disclosed herein.

In examples where aggregate demographic impressions are available,aggregate demographic impressions received or otherwise obtained fromthe example database proprietor 104 are stored in the temporary databaseproprietor database 406 (FIG. 4 ). Similarly, panelist demographicimpressions from the example panelist database 116 (FIG. 1 ) are storedin the temporary panelist database 404 (FIG. 4 ). In some examples, thepanelist demographic impressions are a subset of the aggregate databaseproprietor demographic impressions. As such, the example householdmanager 408 determines whether the panelist demographic impressions areindeed a subset of the aggregate database proprietor demographicimpressions (block 806). If the example household manager 408 determinesthat the panelist impressions are a subset of the aggregate databaseproprietor demographic impressions (block 806: YES), control proceeds toblock 808. If the example household manager 408 determines that thepanelist impressions are not a subset of the aggregate databaseproprietor demographic impressions (block 806: NO), control returns toblock 804, where additional panelist impressions are received. Thiscycle repeats until the panelist impressions are a subset of theaggregate database proprietor demographic impressions.

In some examples, the example OTT-DP matched impressions provideperson-level characteristics (e.g., demographics) that the exampledemographic corrector 110 matches to panelists on an individual level(e.g., at a user level). In these examples, the example demographiccorrector 110 identifies the true error between the demographics of theexample OTT-DP matched impressions and the panelist impressions. Inother examples, the example OTT-DP matched impressions do not provideuser-level demographics, for privacy reasons. In such other examples,the example OTT-DP matched impressions provide household-leveldemographics that the example demographic corrector 110 uses to matchAME panelists to DP household members and to identify errors in thehousehold-level demographics. In some instances, this results inmisattribution because the matching between panelists and DP householdmembers may not be correct.

In other examples, to match DP household-level demographics to AMEpanelist demographics, the example household manager 408 (FIG. 4 )selects a household with corresponding panelist and database proprietordemographic impressions (block 808). In some examples, the examplepanelist database 404 stores household demographic impressions for eachpanelist. In some examples, the example DP database 406 stores householddemographic impressions related to a corresponding panelist household(because the panelists are a subset of the database proprietor users).In some examples, a panelist household demographic impression may bematched to a database proprietor household demographic impression viathe example matched ID discussed in detail above. Thereafter, theexample household manager 408 selects a panelist from the panelisthousehold (block 810). In some examples, once a panelist is selected bythe example household manager 408, the example household manager 408sends the age of the panelist to the example comparator 410. Inadditional examples, the example household manager 408 sends thecorresponding database proprietor household demographic impressions tothe example comparator 410. The example comparator 410, in someexamples, compares the panelist's age to all members of the householddetailed by the database proprietor household demographic impressions(e.g., one member of a panelist household is compared to all members ofthe corresponding database proprietor household) (block 812). Asdisclosed by the examples herein, this is done to detect errors in thedatabase proprietor's demographic impressions.

Once the age comparison is made, the example comparator 410 sends thecomparison data to the example analyzer 412. The example analyzer 412analyzes the age comparison data and at least determines if thepanelist's age is an exact match to any of the database proprietorhousehold members' ages (block 814), if the closest database proprietorhousehold member's age satisfies a threshold (e.g., within three yearsof the panelist's age) (block 816), and/or if there is at least onedatabase proprietor household member with the same gender as thepanelist (block 824).

In some examples, if the example analyzer 412 determines that thepanelist's age is an exact match to any of the database proprietorhousehold members' ages (block 814: YES), then the example analyzer 412determines that the database proprietor demographic impression iscorrect in terms of age and control returns to block 810 to selectanother panelist from the panelist household. However, often thepanelist's age does not match any of the database proprietor householdmembers' ages (block 814: NO).

In some examples, if the example analyzer 412 determines that thepanelist's age does not match any of the database proprietor householdmembers' ages exactly, the example analyzer 412 compares the panelist'sage to the closest matching database proprietor household member's age.If the example analyzer 412 determines that the closest matchingdatabase proprietor household member's age is within the threshold(e.g., a variance of ±3 years, +10 years, etc.) (block 816: YES),control proceeds to block 818. However, if the closest matching databaseproprietor household member's age is not within the threshold (block816: NO), then the example analyzer 412 determines that the databaseproprietor demographic impression is not a correct match to thepanelist's demographic impression and a different panelist householdimpression may be compared to its corresponding database proprietor'shousehold demographic impressions (e.g., control returns to block 808).

In some examples, the threshold discussed above is meant to identifyincorrect recorded demographic information. Incorrect demographicinformation may be recorded for a number of reasons including, withoutlimitation, disinterested user input, minimum demographic requirements,accident, user input error, recording error, or the like. Examplemethods and apparatus disclosed herein identify such incorrect recordeddemographic information from a database proprietor and adjust theincorrect demographic information with panelist demographic information.

At block 818, the example analyzer 412 determines that the databaseproprietor demographic impression is a correct match to the panelist'sdemographic impression and determines the age difference between thematching members. Thereafter, the example analyzer 412 sends theimpression data and an age correction signal to the example adjuster414. In some examples, the example adjuster 414, replaces a databaseproprietor's incorrectly recorded age with a panelist's correctlyrecorded age (block 820). In other examples, the example adjuster 414only stores the age difference for future applications.

Additionally or alternatively in other examples, the example comparator410 compares a panelist's gender to the gender of the household members'genders as recorded by the database proprietor. (Block 824). In manyexamples, gender is less misattributed than age. Therefore, in theexamples described herein, the example analyzer 412 determines whetherthere is at least one database proprietor household member with the samegender as the panelist. (Block 826). If there is at least one databaseproprietor household member with the same gender as the panelist (block826: YES), then the example analyzer 412 determines that the databaseproprietor demographic impression is a correct match to the panelist'sdemographic impression and control proceeds to block 822. However, ifthere is no database proprietor household member with the same gender asthe panelist (block 826: NO), then the example analyzer 412 determinesthat the database proprietor demographic impression is not a correctmatch to the panelist's demographic impression and control returns toblock 808. Accordingly, in some examples, if there is at least oneincorrect match among household members, then it is presumed that thehousehold-level impressions between the panelists and the databaseproprietor do not match. (Blocks 816, 826: NO). As such, the exampleprogram 800 moves to the next household.

When there is at least one database proprietor household member with thesame gender as the panelist (block 826: YES), the example adjuster 414sends the age adjustment data identified in block 818 and block 820 tothe example model trainer 416, in some examples. Based on the panelists'age adjustment data (from block 818 and block 820), the example modeltrainer 416 trains an adjustment model (block 822). After manyiterations of the example program 800, the trained model is applied tothe remaining set of demographic impressions that are not the panelistsubset. In this way, the example methods and apparatus of the presentdisclosure can correct age variations amongst all OTT device users. Ofcourse, while the examples disclose herein refer to age and OTT devices,different demographic data and devices may be utilized without departingfrom the scope of the present disclosure.

When the example analyzer 412 determines a correct match between thedatabase proprietor demographic impression and the panelist'sdemographic impression for both age and gender, the example analyzer 412determines whether there are more panelists in a given household. If theexample analyzer 412 determines that there are more panelists in thegiven household (block 828: YES), control returns to block 810. If thereare no more panelists in the household (block 828: NO), control proceedsto block 830. At block 830, the example analyzer 412 determines whetherthere are any additional panelist households. If the example analyzer412 determines that there are more panelist households (block 830: YES),control returns to block 808. However, if the example analyzer 412determines that there are no more panelist households (block 830: NO),control proceeds to block 832.

At block 832, the example program 800 has ran for a number ofiterations. Therefore, the example model trainer 416 has trained theadjustment model for the same number of iterations using the ageadjustments as a learning (e.g., training) dataset. Accordingly, theadjustment model is ready to be applied to the aggregate demographicimpressions (or remainder thereof if the demographics of the databaseproprietor panelist subset registrants have been corrected by theexample program 800). With reference to FIG. 4 , the example modelapplier 418 receives the adjustment model from the example model trainer416. Additionally, the example model applier 418 (FIG. 4 ) receives theaggregate database proprietor demographic impressions from the exampleDP database 406 (FIG. 4 ).

As discussed above, the adjustment model has the ability to performaccurately on new, unseen examples/tasks (e.g., the aggregatedemographic impressions for the example DP database 406) after havingexperienced a learning dataset (e.g., the iterative age adjustments forthe panelist subset of the aggregate demographic impressions). In thismanner, example model applier 418 (FIG. 4 ) applies the adjustment modelto the entirety of the database proprietor demographic impressions tocorrect the ages (block 832). In some examples, the example modelapplier 418 (FIG. 4 ) outputs the corrected aggregate demographicimpressions. Thereafter, the example program 800 ceases. Of course,while the example program 800 discusses age identification, correction,model training, etc., other demographic information may be correctedsimilarly using the example program 800. Thus, the example program 800should not be limited in this manner.

FIGS. 9A-9C depict another flow diagram representative of examplemachine readable instructions that may be executed to implement theexample demographic corrector 110 of FIGS. 1 and 4 . The example program900 begins at block 902. The example demographic impression receiver 402receives or otherwise obtains aggregated demographic impressions fromthe example database proprietor 104 (FIG. 1 ) (block 902) and panelistdemographic impressions from the example panelist database 116 (FIG. 1 )(block 904). In some examples, aggregate demographic impressions areunavailable. In such examples, demographic impressions are created withthe example viewership assigner 114, as disclosed herein.

In some examples, the example demographic impression receiver 402distributes the aggregated demographic impressions from the exampledatabase proprietor 104 collected at block 902 to the example DPdatabase 406 (FIG. 4 ). In other examples, the example demographicimpression receiver 402 (FIG. 4 ) distributes the panelist demographicimpressions from the example panelist database 116 collected at block904 to the internal panelist database 404 (FIG. 4 ). From here, theexample household manager 408 (FIG. 4 ) retrieves demographicimpressions from the example panelist database 404 and/or the example DPdatabase 406. In some examples, the example household manager 408initially determines whether the panelist impressions are a subset ofthe DP aggregate demographic impressions (block 906). In some examples,the panelist impressions correspond to panelists with OTT devices andtherefore are likely a subset of the DP aggregate demographicimpressions. If the example household manager 408 determines that thepanelist impressions are a subset of the DP aggregate demographicimpressions (block 906: YES), control proceeds to block 908. However, iffor some reason the example household manager 408 determines that thepanelist impressions are not a subset of the DP aggregate (block 906:NO), control returns to block 804 to find panelist impressions that area subset of the DP aggregate demographic impressions. At block 908, ahousehold having a number of impressions associated therewith isselected by the example household manager 408.

Once a household is selected, control proceeds to block 920, shown inFIG. 9B. The example household manager 408, in some examples, organizesa first list of DP household (“HH”) members having a first or unknowngender (e.g., the first gender is female in some examples) (block 920).In additional examples, the example household manager 408 organizes asecond list of panelist HH members having the first gender (block 922).In some examples, the DP HH members are registrants of the exampledatabase proprietor 104 (FIG. 1 ) that corresponds to AME panelistregistrants. In such examples, the example database proprietor 104 hasdemographic data for the DP HH members and the example panelist database116 (FIG. 1 ) has demographic data for the Panelist HH members. Theexample household manager 408 thereafter constructs a first matrix usingthe first and second lists. The example household manager 408 uses theages of the DP HH members from the first list as column legends and theages of the panelist HH members from the second list as row legends(block 924). Thereafter, the example comparator 410 compares each columnlegend to each row legend and calculates the absolute difference (e.g.,absolute difference=column−row, if column>row; absolutedifference=row−column, if row>column) between them, in some examples, topopulate a plurality of cells (e.g., the corresponding intersections ofthe columns and rows) of the first matrix (block 926).

Thereafter, in some examples, the example analyzer 412 analyzes thepopulated matrix from block 926 to find the cell with the lowest value(e.g., minimum absolute difference) (block 928). Further, the exampleanalyzer 412 determines whether the value found in block 928 is below(e.g., less than) a threshold (e.g., 10 years) (block 930). If theexample analyzer 412 determines the value from block 928 is above thethreshold (block 930: NO), control proceeds to block 950. If the exampleanalyzer 412 determines the value from block 928 is below the threshold(block 930: YES), control proceeds to block 932.

In some examples, one or more of the DP HH members have no age dataavailable. In these examples, the DP HH members with no age data (e.g.,a corresponding column in the matrix) are ignored until the rest of theDP HH members are matched as described herein. In some examples, asingle panelist HH member and DP HH member remain in the matrix. In suchexamples, the unknown age for the DP HH member is corrected with the ageof the remaining panelist HH member's age. In other examples, multiplepanelist HH members and multiple ageless DP HH members remain in thematrix after an iteration. In such examples, additional information(e.g., media data, timestamp data, etc.) may be required to predict orotherwise match ageless DP HH members to the panelist HH members.

At block 950, the example analyzer 412 determines that the DP HH membercorresponding to the column legend from the first list is not a match tothe Panelist HH member corresponding to the row legend from the secondlist. The example analyzer 412 labels each member as not matched, insome examples, and control proceeds to block 944. In some examples, thenon-matched HH members are analyzed at block 960, as discussed inconnection with FIG. 9C. Additionally or alternatively, the non-matchedHH members are sent to the example demographic predictor 114 (FIG. 6 ),in some examples.

At block 944, the example analyzer 414 (FIG. 4 ) determines whetherthere is another gender for which to perform analysis (e.g., if a matrixwas created and analyzed for males, then a matrix still needs to becreated and analyzed for females and vice-a-versa). If the exampleanalyzer 414 determines that there is another gender for which toperform analysis (block 944: YES), control proceeds to block 946.

At block 946, the example household manager 408 organizes a first listof DP household (“HH”) members having a second or unknown gender. Inadditional examples, the example household manager 408 organizes asecond list of panelist HH members having the second gender (block 948).Thereafter, control returns to block 924, where the example householdmanager 408 constructs a second matrix using the first and second lists.The example household manager 408 uses the ages of the DP HH membersfrom the first list as column legends and the ages of the panelist HHmembers from the second list as row legends (block 924).

When the example analyzer 412 determines the value from block 928 isbelow the threshold (block 930: YES), the example analyzer 412determines that the DP HH member corresponding to the column legend fromthe first list is a match to the Panelist HH member corresponding to therow legend from the second list (block 932). In some examples, theexample analyzer 412 identifies the age difference between the matchedmembers (e.g., the value from block 928) and sends the age difference tothe example adjuster 414 (block 934). In some examples, the exampleadjuster 414 adjusts the DP HH member's age according to the agedifference obtained from the example analyzer 412 at block 934 (block936). At block 936, the example adjuster 414 relays the age adjustmentto the example model trainer 416, which trains an adjustment model basedon the age adjustment (block 938). After the adjustment model has beentrained with the age adjustment from block 938, the example analyzer 412removes the column and row corresponding to the DP HH member and thepanelist HH member matched in block 932 from the matrix (block 940). Ifthe example analyzer 412 determines that there are additional cells leftin the matrix (block 942: YES), control returns to block 928 where theremaining cells are analyzed by the example analyzer 412 to find thecell with the lowest, or minimum value. This process continues for asmany iterations as there are cells in the matrix.

If the example analyzer 412 determines that there are no more cells inthe matrix after block 940 (block 942: NO), control proceeds to block944. At block 944, the example analyzer 414 determines whether there isanother gender to perform analysis for (e.g., if a matrix was createdand analyzed for males, then a matrix still needs to be created andanalyzed for females and vice-a-versa). If the example analyzer 414determines that there is another gender for which to perform analysis(block 944: YES), control proceeds to block 946. If the example analyzer414 determines that analysis has been done for all genders (block 944:NO), then control proceeds to block 910, shown in FIG. 9A.

As further shown in FIG. 9C, block 960 accounts for the HH membersnot-matched previously. The example household manager 408 furtherconstructs a third matrix using the ages from non-matched DP HH members(e.g., from block 950) as column legends and the ages from non-matchedpanelist HH members as row legends (e.g., from block 950) similarly tothe process described in connection to block 924 (block 960).

Once the example processor 408 constructs the third matrix, the examplecomparator 410 compares and calculates the absolute difference betweenthe column legends and the row legends to populate the cells of thethird matrix (block 962). In some examples, this is similar to theprocess described in connection to block 926. The example analyzer 412then identifies the cell with the lowest or minimum value (block 964).Subsequently, the example analyzer 412 determines whether the valueidentified in block 964 is below (e.g., less than) a threshold (e.g., 10years) (block 966). If the example analyzer 412 determines that thevalue identified in block 964 is below the threshold (block 966: YES),control proceeds to block 968.

If the example analyzer 412 determines that the value identified inblock 964 is above the threshold (block 966: NO), control proceeds toblock 980.

When the example analyzer 412 determines that the value identified inblock 964 is below the threshold (block 966: YES), the example analyzer412 determines the DP HH member corresponding to the column legend is amatch to the panelist HH member corresponding to the row legend (block968). In some examples, the example analyzer 412 identifies the agedifference between the matched DP HH member and panelist HH member(e.g., the value from block 964) and sends the age difference to theexample adjuster 414 (block 970). Thereafter, the example adjuster 414adjusts the DP HH member's age according to the age difference obtainedfrom the example analyzer 412 at block 970 (block 972). The exampleadjuster 414 relays the age adjustment to the example model trainer 416,which trains an adjustment model based on the age adjustment (block974). After the adjustment model has been trained with the ageadjustment from block 974, the example analyzer 412 removes the columnand row corresponding to the DP HH member and the panelist HH membermatched in block 968 from the third matrix (block 976). If there are nomore cells in the third matrix after block 976 (block 978: NO), thencontrol proceeds to block 910, shown in FIG. 9A. If the example analyzer412 determines that there are additional cells left in the third matrix(block 978: YES), control returns to block 964 where the remaining cellsare analyzed by the example analyzer 412 to find the cell with thelowest, or minimum value. This process continues for as many iterationsas there are cells in the third matrix.

At block 980, the example analyzer 414 determines whether there are morecolumns than rows in the third matrix. If the example analyzer 412determines that there are more columns than rows in the third matrix(block 980: YES), control proceeds to block 982. If the example analyzer412 determines that there are not more columns than rows in the thirdmatrix (e.g., the same amount of rows and columns, or less columns thanrows; block 980: NO), control proceeds to block 984. At block 984, theexample analyzer 414 determines whether there are more rows than columnsin the third matrix. If the example analyzer 412 determines that thereare more rows than columns in the third matrix (block 984: YES), controlproceeds to block 986. If the example analyzer 412 determines that thereare not more rows than columns (e.g., same amount of rows and columns;block 984: NO), control proceeds to block 988.

When the example analyzer 412 determines that there are more columnsthan rows in the third matrix (block 966: NO; block 980: YES), then theexample adjuster 414 assigns an “EXTRA” label to the non-matched DP HHmember(s) (block 982). In some examples, this indicates that thedatabase proprietor is accounting for a person, or persons, that shouldnot be accounted for within a particular household (e.g., a guest,friend, or other non-resident). In such examples, this indication isbased upon the accuracy of the panelist impression data, including thenumber of residents within a panelist's household (e.g., the panelisthousehold has less members than the corresponding database proprietorhousehold; therefore the DP HH has extra members). Next, the examplemodel trainer 416 trains an adjustment model based on the assignments bythe example adjuster 414 (block 990). Thereafter, control proceeds toblock 910, shown in FIG. 9A.

When the example analyzer 412 determines that there are more rows thancolumns in the third matrix (block 966: NO; block 980: NO; block 984:YES), then the example adjuster 414 adds the panelist(s) associated withthe extra row(s) to the DP HH members and assigns a “MISSING” label tothe panelist(s) (block 986). In some examples, this indicates that thedatabase proprietor is not accounting for a person, or person, that theyshould be accounting for within a particular household (e.g., a minor,an elder, or another resident). In these examples, this indication isbased upon the accuracy of the panelist impression data, including thenumber of residents within a panelist's household (e.g., the panelisthousehold has more members than the corresponding database proprietorhousehold; therefore a panelist is missing from the DP HH). Next, theexample model trainer 416 trains an adjustment model based on theassignments by the example adjuster 414 (block 990). Thereafter, controlproceeds to block 910, shown in FIG. 9A.

In some examples, when the example analyzer 412 determines that thereequal rows and columns in the third matrix (block 966: NO; block 980:NO; block 984: NO), then the example analyzer 412 assigns an “EXTRA”label to the non-matched DP HH member(s). In other examples, the exampleadjuster 414 adds the panelist(s) associated with the remaining row(s)to the DP HH members and assigns “MISSING” label(s) to the panelist(s).Such examples indicate that the database proprietor is accounting for aperson, or persons, that should not be accounted for within a particularhousehold (e.g., a guest, friend, or other non-resident) and/or thedatabase proprietor is not accounting for a person, or persons, that theDP should be accounting for within a particular household (e.g., aminor, an elder, or another resident). Next, the example model trainer416 trains an adjustment model based on the assignments by the exampleadjuster 414 (block 990). Thereafter, control proceeds to block 910,shown in FIG. 9A. As described above, the example adjustment modeltrained by the example model trainer 416 at block 990 accounts forvariances in the members of a particular household.

Returning to block 910, as shown in FIG. 9A, the example program 900 hasevaluated a household and has matched DP HH members to panelist HHmembers or has labeled the non-matching members accordingly, in someexamples. Thereafter, the example analyzer 412 (FIG. 4 ) determines ifthere are additional households with impressions. If the exampleanalyzer 412 determines that there are additional households withimpressions (block 910: YES), control returns to block 908 where anotherhousehold having a number of impressions associated therewith isselected by the example household manager 408 (FIG. 4 ). If the exampleanalyzer 412 determines there are no additional households to analyze(block 910: NO), then the adjustment model trained by the example modeltrainer 416 (FIG. 4 ) is sent to the example model applier 418 (FIG. 4 )(block 912).

At block 912, the example model applier 418 applies the adjustment modelto the DP aggregate demographic impressions from the example DP database406. In some examples, the adjustment model corrects any agediscrepancies in the DP aggregate demographic impressions using themodel trained by the subset of panelist impressions at block 938. Inother examples, the adjustment model corrects gender discrepancies inthe DP aggregate demographic impressions using the model trained by thesubset of panelist impressions at block 974. In even further examples,the adjustment model corrects HH member variances when there are missingand/or extra members accounted in the DP aggregate demographicimpressions.

FIG. 10 is a flow diagram representative of example machine readableinstructions that may be executed to implement the example viewershipassigner 112 of FIGS. 1 and 5 . In the illustrated example, the exampleviewership assigner 112 uses the corrected demographics from the exampledemographic corrector 110 to create demographic impressions assigned tohousehold members. In the illustrated example, the example OTTdemographic impression receiver 502 receives household impressioninformation and corrected demographics from the example demographiccorrector 110. The example OTT database 504 associates and stores thehousehold impression information and corrected demographics formanipulation by an example program 1000. The example program 1000assigns particular household members to demographic impressionspreviously assigned to the household.

The example program 1000 begins at block 1001. At block 1001, theexample HH member manager 510 determines whether, for a first householdassociated with an impression of a media presentation, the size of thefirst household is equal to one. If only one member exists in the firsthousehold (block 1001: YES), then the example OTT viewership assigner516 associates the first household impression with the single householdmember (block 1002) because there are no other household members thatcould possibly have viewed the media presentation besides the singlehousehold member. However, when size of the first household is not equalto one (block 1001: NO), control proceeds to block 1004 in order todetermine which household member actually viewed the media presentation.

In order to determine which household member viewed the mediapresentation, the example viewership assigner 112 utilizes conditionalprobabilities based on donor (e.g., panelist and/or historical viewingdata) household data. Therefore, the example donor data receiver 506acquires donor data via the example network 122 (block 1004). Theexample donor data receiver 506 stores the example donor data in theexample donor database 508. The example viewership analyzer 512 uses aplurality of variables to find donor household information in theexample donor database 508 that match with data related to the firsthousehold. Examples of matching variables include, without limitation,demographics (male ages 2-12, 13-4, 15-17, etc., female ages 40-44,50-54, etc.), time of day (e.g., prime time, late afternoon/earlyevening, late evening, weekday morning, weekday daytime, weekenddaytime, etc.), specific genre (e.g., musical drama, feature film,comedy variety, children, etc.), HH size, number of adults, number ofkids, etc. The example viewership analyzer 512 refines the donor databased on the matching variables for a donor household that has similarmatching variables with the first household (block 1006). If no matchingdonor households are located (block 1008: NO), then control proceeds toblock 1010.

At block 1010, there are no donor households that match with all thematching variables associated with the first household. Accordingly, theexample viewership adjuster 514 rolls up the variables, such that amatch may be found. In the illustrated example, rollup refers toiteratively removing variables until a match can be found. In someexamples, initially a donor household must match all matching variablesassociated with the first household. After the example viewershipadjuster 514 rolls up the variables a first time, only demographics,time of day, general genre (e.g., children, comedy, drama, news, sports,etc.), HH size, number of adults, and number of kids must match. Afterthe example viewership adjuster 514 rolls up the variables a secondtime, only demographics, time of day, general genre, HH size, and numberof kids must match. After the example viewership adjuster 514 rolls upthe variables a third time, only demographics, time of day, generalgenre, and number of kids must match. After the example viewershipadjuster 514 rolls up the variables a fourth time, only demographics,time of day, general genre, and the presence of kids (e.g., binary yesor no) must match. After the example viewership adjuster 514 rolls upthe variables a fifth time, only demographics, time of day, and generalgenre must match, etc. After the example viewership adjuster 514 rollsup the matching variables (block 1010), control returns to block 1006where the example viewership analyzer 512 refines the donor data basedon the rollup variables.

However, when a matching donor household is found (block 1008: YES),control proceeds to block 1012. At block 1012, the example viewershipanalyzer 512 selects a first demographic bucket (e.g., male ages 18-20).As discussed in conjunction with the demographic corrector 110, theexample viewership analyzer 512 has access to the actual demographicbuckets that exist within the first household. Therefore, the exampleviewership analyzer 512 selects a demographic bucket that actuallyexists in the first household, rather than an arbitrary demographicbucket. For the first demographic bucket, the example viewershipanalyzer 512 identifies a first time that a person in the donorhousehold matching the first demographic bucket previously viewed themedia presentation (e.g., based on panelist viewing history) (block1014). Additionally, the example viewership analyzer 512 identifies asecond time that all members in the donor household previously viewedthe media presentation (block 1016). Thereafter, the example viewershipanalyzer 512 divides the first time by the second time to determine aprobability that the first demographic bucket viewed the mediapresentation (block 1018). At block 1020, the example viewershipanalyzer 512 determines if there are additional demographic buckets forwhich to determine probabilities. If there are additional demographicbuckets (block 1020: YES), control returns to block 1012. If there areno additional demographic buckets (block 1020: YES), control proceeds toblock 1022.

At block 1022, all probabilities for the existing demographic bucketswithin the first household have been determined. Thereafter, the exampleviewership analyzer 512 searches for a donor viewing event. The exampleviewership analyzer 512 refines the donor data similarly to block 1006using matching variables such as, for example, a time of day, genre,household size, number of adults, and number of children (e.g.,demographic matching is left out in some examples) (block 1022). Theexample viewership analyzer 512 determines if there are more than onematching viewing event located in the example donor database 508 (block1024). If there is not more than one viewing event (block 1024: NO), theexample viewership analyzer 512 determines if there is just one matchingviewing event (block 1026). If there is not one matching viewing event(block 1026: NO), then no viewing events were found and control proceedsto block 1028. At block 1028, the example viewership adjuster 514 rollsup the matching variables as described above and control returns toblock 1022, where the example viewership analyzer 512 refines the donordata based on the rollup variables.

If there are more than one matching viewing event (block 1024: YES),then control proceeds to block 1030 to determine which viewing event touse as a donor viewing event. The example viewership analyzer 512 ranksboth the donor household members and the first household membersaccording to their respective probabilities determined in block 1018(block 1030). The example viewership analyzer 512 then determines theabsolute difference between the probabilities of the household membersfrom the first household and the probabilities of the donor householdmembers (e.g., historical probability data) (block 1032). For example,the absolute difference between the first ranked household member (e.g.,ranked via block 1030) of the first household and the first rankedhousehold member of the donor household is determined, the absolutedifference between the second ranked household member of the firsthousehold and the second ranked household member of the donor householdis determined, etc. The example viewership analyzer 512 adds thedifferences determined via block 1032 for each donor viewing event(block 1034). The example viewership analyzer 512 then selects the donorviewing event associated with the lowest summation determined via block1034 as the donor viewing event for the first household (block 1036).

After selecting the donor viewing event via block 1036 or if there isjust one matching viewing event (block 1026: YES), control proceeds toblock 1038. At block 1038, the example OTT viewership assigner 516imputes donor viewing (e.g., known historical data) to respective firsthousehold members. For example, if the first ranked household member ofthe donor household (e.g., ranked via block 1030) had viewed the mediathen the example OTT viewership assigner 516 assigns the first rankedhousehold member of the first household as having viewed the mediapresentation. Thereafter, the example program 1000 ceases. Of course,the example program 1000 may be ran again for another demographicimpression.

FIG. 11 is another flow diagram representative of example machinereadable instructions that may be executed to implement the exampleviewership assigner 112 of FIGS. 1 and 5 . In some examples, assigningviewership to a particular household member is based on the mediapresented, the time of the media presentation, and/or householdcharacteristics of a particular household. An example program 1100 forimplementing the example viewership assigner 112 is shown in FIG. 9 andstarts at block 1102. At block 1102, the example donor data receiver 506(FIG. 5 ) receives times (and media at those times) that the donorhousehold members were using the television. The example viewershipassigner 112, in some examples, receives this information from the AME's106 (FIG. 1 ) television meters. Alternatively, the media and timeinformation may be processed and stored prior to use by the exampleviewership assigner 112.

After receiving the television media and times, the donor data receiver506 stores the media and time information into a donor database 508(FIG. 5 ) and control proceeds to block 1104. Next, the example OTTdemographic impression receiver 502 (FIG. 5 ) receives the correcteddemographic impressions (e.g., corrected age and gender for eachdemographic impression) from the example demographic corrector 110 (FIG.4 ). The corrected demographic impressions may additionally include atleast the times that the recipient household used the example OTT device102 (FIG. 1 ) along with the corrected demographic impressions (block1104). In some examples, the example OTT demographic impression receiver502 stores at least the example OTT device 102 use times in the exampleOTT database 504 (FIG. 5 ).

The example methods and apparatus disclosed herein utilizes historicalextraction to predict the media of the example OTT device 102 at thetimes the example OTT device 102 was used. In some examples, thehousehold member manager 510 extracts the viewing history and media oftelevision and PC views obtained from the donor data receiver 506. Thedonor data receiver 506 receives media and timestamps of televisionmedia for particular panelists from panelist metering methods (e.g.,watermarking media, encoding media, attaching cookies to media, etc.) insome examples. In other examples, the media and timestamps of televisionmedia is estimated or predicted from statistical models based uponpanelist metering methods. The media and timestamps are retrieved overthe example network 122 in some examples.

In some examples, the example HH member manager 510 predicts that theusage of the example OTT device 102 is similar to the historical usage(e.g., media and time associated with a particular viewer) of thetelevision or PC. For example, if a particular viewer frequently watchedmusic videos on MTV and YouTube.com, then that particular viewer wouldbe likely to use the example OTT device 102 to watch music videosthrough particular device channels and/or on a YouTube application onthe example OTT device. The example viewership assigner 112 has accessto television and PC media and viewing times of that media. Thetelevision and PC media includes genre, program name, program rating,advertisements, timestamps, metadata, and other information in someexamples.

In some examples, the example viewership assigner 112 accounts forco-viewing (e.g., more than one household member is viewing theparticular media for which the impression represents.). Thus, theexample household member manager 510 has the option to account forco-viewing (block 1106). If co-viewing is to be accounted for (block1106: YES), control proceeds to block 1112 and the household membermanager 510 sends data to the example viewership analyzer 512. Otherwise(block 1106: NO), control proceeds to block 1108 the household membermanager 510 sends data to the example OTT viewership assigner 510.

When co-viewing is not required (block 1106: NO), then the example OTTviewership assigner 510 determines which household member is likelyusing the example OTT device during the time the household used theexample OTT device (block 1108). As discussed above, the exampleviewership assigner 510 utilizes television and PC media in someexamples, to match household members to a particular OTT viewing (e.g.,if, historically, adult females watch the Lifetime network on TV, it islikely that similar OTT media is also watched by adult females). Inother examples, the viewership assigner 510 utilizes the time of thetelevision and/or PC media to match household members to a particularOTT device viewing (e.g., if, historically, media watched from 3-5 P.M.is watched by children after school, OTT media watched during the sametime is likely watched by children). Once the viewership assigner 510has determined which particular household member is using the exampleOTT device, then the viewership assigner 510 assigns viewership of aparticular impression associated with the time the example OTT devicewas used to the particular household member (block 1110). Thereafter theexample program 1100 ceases. Of course, the example program 1100 may berun again.

In some examples co-viewing is to be accommodated (block 1106: YES). Ofcourse, many methods may be used to account for co-viewing withoutdeparting from the scope of the present disclosure. In backwardelimination examples, shown in FIG. 11 , the example viewership analyzer512 preliminarily assigns to all members of a household viewership of aparticular OTT media impression (block 11). From there, the exampleviewership analyzer 512 selects a household member for an eliminationanalysis (block 1114). In some examples, the example viewership analyzer512 analyzes and compares the media and time of the example OTT mediaimpression to the household member's television and PC media and viewinghistory (block 1116). If the example viewership analyzer 512 determinesthat the media of the example OTT media impression conflicts with the HHmember's television and PC media history (block 1118: YES), controlproceeds to block 1120. If the example viewership analyzer 512determines that the media of the example OTT media impression does notconflict with the HH member's television and PC media history (block1118: NO), control proceeds to block 1122.

When the example viewership analyzer 512 determines that the media ofthe example OTT media impression conflicts with the HH member'stelevision and PC media history (block 1118: YES), the exampleviewership adjuster 514 removes viewership of the particular OTT mediaimpression from the HH member (block 1120). If the example viewershipanalyzer 512 determines that additional HH members exist (block 924:YES), then the next HH member is selected at block 11 and controlproceeds from there. However, if the example viewership analyzer 512determines that no additional HH members exist (block 1124: NO), thenthe example OTT viewership assigner 510 permanently assigns theremaining HH members viewership of the particular OTT media impression(block 1126). Thus, the example program 1100 assigns particularhousehold members (including co-viewers) to impressions such thatdemographics for corresponding impressions from OTT devices areassociated with relatively higher accuracies. Thereafter the exampleprogram 1100 ceases. Of course, the example program 1100 may be runagain.

When the example viewership analyzer 512 determines that the media ofthe example OTT media impression does not conflict with the HH member'stelevision and PC media history (block 1118: NO), the example viewershipanalyzer 512 determines whether the example OTT impression viewing timeconflicts with the HH member's television and PC viewing history (block1122). If the example viewership analyzer 512 determines that theexample OTT impression viewing time conflicts with the HH member'stelevision and PC viewing history (block 1122: YES), control returns toblock 1120. In some examples, the example viewership adjuster 514removes viewership of the particular OTT media impression from the HHmember when the example OTT impression viewing time conflicts with theHH member's television and PC viewing history. Thereafter, the exampleviewership analyzer 512 determines if there are any additional HHmembers. If the example viewership analyzer 512 determines thatadditional HH members exist (block 1124: YES), then the next HH memberis selected at block 1114 and control proceeds from there. If theexample viewership analyzer 512 determines that no additional HH membersexist (block 1124: NO), then the example OTT viewership assigner 510permanently assigns the remaining HH members to the particular OTT mediaimpression (block 1126). Thereafter, the example program 1100 ceases. Ofcourse, the example program 1100 may be run again.

If the example viewership analyzer 512 determines that the example OTTimpression viewing time does not conflict with the HH member'stelevision and PC viewing history (block 1122: NO), then the exampleviewership analyzer 512 checks if there are any more HH members left toanalyze (block 1124). If the example viewership analyzer 512 determinesthat there are additional HH members (block 1124: YES), the controlreturns to block 1114. If the example viewership analyzer 512 determinesthat no additional HH members exist (block 1124: NO), then the exampleOTT viewership assigner 510 permanently assigns the remaining HH membersto the particular OTT media impression (block 1126). Thereafter, theexample program 1100 ceases. Of course, the example program 1100 may berun again. As discussed above, additional or alternative methods ofassigning viewership to a household member may be utilized withoutdeparting from the scope of the present disclosure.

FIG. 12 is a flow diagram representative of example machine readableinstructions that may be executed to implement the example demographicpredictor 114 of FIGS. 1 and 6 . The example demographic predictor 114,in some examples, predicts the demographics of DP HH members not matchedwith OTT device 102 user data 204 (FIG. 2 ). Presumptively, ifdemographics from the example database proprietor 104 (FIG. 2 ) is notmatched with user data from the example OTT device 102 (FIG. 2 ), thenthe corresponding DP HH members using an OTT device will not havedemographic data available to correct by the example demographiccorrector 110. Therefore, the example demographic predictor 114 utilizesa plurality of methods to make up for any deficits in demographic dataas further described below.

An example program 1200 begins at block 1202, wherein the exampleprediction manager 610 determines whether to use a scaling method topredict the demographics of DP HH members not matched with OTT device102 user data 204 (e.g., OTT-DP non-matched impressions). If the exampleprediction manager 610 determines a scaling method is to be used (block1202: YES), control proceeds to block 1204.

At block 1204, the example prediction manager 610 determines whether toscale the example OTT-DP matched impressions to account for thenon-matched impressions. If the example prediction manager 610determines to scale the example OTT-DP matched impressions (block 1204:YES), then the example prediction manager receives the example OTT-DPmatched impressions from the example OTT-DP matched impressions database604 (block 1206). After receiving the example OTT-DP matchedimpressions, the prediction manager 610, in some examples, determineswhether to apply a linear scaling method on the example OTT-DP matchedimpressions (block 1210). If the prediction manager 610 determines alinear scaling method is to be applied (block 1210: YES), controlproceeds to block 1212. A linear scaling method is useful where theunknown or non-matched OTT-DP impressions are presumptively similar tothe example OTT-DP matched impressions for all demographic buckets. Forexample, when 95% of the example OTT-DP matched impressions cover adiverse plurality of demographics (e.g., all demographic buckets areaccounted for), the example OTT-DP matched impressions can be linearlyscaled to 100%. At block 1212, the example linear scaler 612 scales theexample OTT-DP matched impressions linearly to 100%. For example, theunknown or non-matched OTT-DP impressions are replaced with impressionsstatistically representative of the example OTT-DP matched impressions,according to a linear scale. Thereafter, the example modeler 616 trainsa prediction model based on the linear scaling at block 1222.

If the prediction manager 610 determines a linear scaling method is notsufficient (block 1210: NO), control proceeds to block 1214. In someexamples, the prediction manager 610 applies a differential scalingmethod when a linear scaling method is insufficient. A differentialscaling method is useful where the unknown or non-matched OTT-DPimpressions are likely not similar to the example OTT-DP matchedimpressions for each demographic bucket. For example, when 95% of theexample OTT-DP matched impressions do not cover a diverse plurality ofdemographics (e.g., one or more demographics are not accounted for, oneor more demographics have different match rates, etc.), linearly scalingwill not account for the lack of diversity. Thus at block 1214, theexample differential scaler 614 scales the example OTT-DP matchedimpressions differentially (e.g., according to a panelist data matchrate of each demographic bucket as discussed in conjunction with FIG. 6). Thereafter, the example modeler 616 trains a prediction model basedon the differential scaling at block 1222.

If the example prediction manager 610 determines not to scale theexample OTT-DP matched impressions (block 1204: NO), control proceeds toblock 1208. At block 1208, the example OTT demographic impressionreceiver 602 receives the corrected demographics from the exampledemographic corrector 110. The prediction manager 610, in some examples,determines whether to apply a linear scaling method on the correcteddemographics from the demographic corrector 110 (block 1210). If theprediction manager 610 determines a linear scaling method is to beapplied (block 1210: YES), control proceeds to block 1212. A linearscaling method is useful where the unknown or unmatched demographics arepresumptively similar to the corrected demographics. For example, when95% of the corrected OTT demographic impressions cover a diverseplurality of demographics (e.g., all demographic buckets are accountedfor), the corrected demographics can be linearly scaled to 100%. Atblock 1212, the example linear scaler 612 scales the correcteddemographics linearly to 100%. For example, the unknown or unmatcheddemographics are replaced with demographics statistically representativeof the corrected demographics, according to a linear scale. Thereafter,the example modeler 616 trains a prediction model based on the linearscaling at block 1222.

If the prediction manager 610 determines a linear scaling method is notsufficient (block 1210: NO), control proceeds to block 1214. In someexamples, a differential scaling method is applied when a linear scalingmethod is insufficient. A differential scaling method is useful wherethe unknown or unmatched demographics are likely not similar to thecorrected demographics. For example, when 95% of the corrected OTTdemographic impressions do not cover a diverse plurality of demographics(e.g., one or more demographics are not accounted for), linearly scalingwill not account for the lack of diversity. Thus at block 1214, theexample differential scaler 614 scales the corrected demographicsdifferentially (e.g., according to a panelist data match rate of eachdemographic bucket as discussed in conjunction with FIG. 6 ).Thereafter, the example modeler 616 trains a prediction model based onthe differential scaling at block 1222.

If the example prediction manager 610 determines a scaling method is notsufficient (block 1202: NO), control proceeds to block 1216. At block1216, the example prediction manager 610 receives demographicinformation associated with a HH member's IP address from the IP addressdemographic receiver 608. The IP address demographics come from anadditional or alternative database proprietor (e.g., eXelate), differentfrom the example database proprietor 104, in some examples, because theexample database proprietor 104 lacks demographic data for the HHmember. Thereafter, the example prediction manager 610 matches thedemographics received by the example IP address demographic receiver 608to the non-matched impressions (block 1218). In some examples, theprediction manager 610 matches the demographics to the non-matchedimpressions similarly to the aforementioned example demographiccomparator 107 in the example registration phase in FIG. 2 . In otherexamples, the prediction manager 610 matches the demographics to thenon-matched impressions using the IP address associated with the HHmember and the example OTT device's IP address or the IP address of aninternet router that the example OTT device 102 accesses.

Thereafter, in some examples, the prediction manager 610 associates thedemographics with the non-matched impressions similarly to how theexample database proprietor 104 (FIG. 3 ) creates the exampledemographic impressions 306 (FIG. 3 ) as shown in the example impressioncollection phase in FIG. 3 . (Block 1220). The example modeler 616subsequently trains a prediction model based on the demographicassociation at block 1222. The example modeler 616 applies any modelstrained (e.g., from linear scaling, differential scaling, and/or IPaddress demographic association) to the aggregate demographicimpressions (block 1222).

FIG. 13 is a block diagram of an example processor platform 1300 capableof executing the instructions of FIGS. 7, 8, 9A-9C, 10, 11, and 12 toimplement the example apparatus 108 of FIG. 1 , the example demographiccorrector 110 of FIG. 4 , the example viewership assigner 112 of FIG. 5, and the demographic predictor 116 of FIG. 6 . The processor platform1300 can be, for example, a server, a personal computer, a mobile device(e.g., a cell phone, a smart phone, a tablet such as an iPad™), apersonal digital assistant (PDA), an Internet appliance, a DVD player, aCD player, a digital video recorder, a Blu-ray player, a gaming console,a personal video recorder, a set top box, or any other type of computingdevice.

The processor platform 1300 of the illustrated example includes aprocessor 1312. The processor 1312 of the illustrated example ishardware. For example, the processor 1312 can be implemented by one ormore integrated circuits, logic circuits, microprocessors or controllersfrom any desired family or manufacturer.

The processor 1312 of the illustrated example includes a local memory1313 (e.g., a cache). The processor 1312 of the illustrated example isin communication with a main memory including a volatile memory 1314 anda non-volatile memory 1316 via a bus 1318. The volatile memory 1114 maybe implemented by Synchronous Dynamic Random Access Memory (SDRAM),Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory(RDRAM) and/or any other type of random access memory device. Thenon-volatile memory 1316 may be implemented by flash memory and/or anyother desired type of memory device. Access to the main memory 1314,1316 is controlled by a memory controller. The processor 1312 of theillustrative example further includes the example demographic corrector110 (FIG. 4 ), the example viewership assigner 112 (FIG. 5 ), and theexample demographic predictor 114 (FIG. 6 ) as disclosed herein.

The processor platform 1300 of the illustrated example also includes aninterface circuit 1320. The interface circuit 1320 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), and/or a PCI express interface.

In the illustrated example, one or more input devices 1322 are connectedto the interface circuit 1320. The input device(s) 1322 permit(s) a userto enter data and commands into the processor 1312. The input device(s)can be implemented by, for example, an audio sensor, a microphone, acamera (still or video), a keyboard, a button, a mouse, a touchscreen, atrack-pad, a trackball, isopoint and/or a voice recognition system.

One or more output devices 1324 are also connected to the interfacecircuit 1320 of the illustrated example. The output devices 1324 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay, a cathode ray tube display (CRT), a touchscreen, a tactileoutput device, a light emitting diode (LED), a printer and/or speakers).The interface circuit 1320 of the illustrated example, thus, typicallyincludes a graphics driver card, a graphics driver chip or a graphicsdriver processor.

The interface circuit 1320 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem and/or network interface card to facilitate exchange of data withexternal machines (e.g., computing devices of any kind) via a network1326 (e.g., an Ethernet connection, a digital subscriber line (DSL), atelephone line, coaxial cable, a cellular telephone system, etc.).

The processor platform 1300 of the illustrated example also includes oneor more mass storage devices 1328 for storing software and/or data.Examples of such mass storage devices 1328 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, RAIDsystems, and digital versatile disk (DVD) drives. The mass storagedevices 1328 include the example panelist database 404, the example DPdatabase 406, the example OTT database 504, and the example donordatabase 508.

The coded instructions 1332 of FIGS. 7, 8, 9A-9C, 10, 11, and 12 may bestored in the mass storage device 1328, in the volatile memory 1314, inthe non-volatile memory 1316, and/or on a removable tangible computerreadable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that the above disclosedmethods and apparatus advantageously discover errors in demographicsassociated with database proprietors in connection with impressionsassociated with OTT devices, correct the errors in the demographics,assign viewership to impressions based on the corrected demographics,predict demographics for impressions lacking associated demographics,and, more generally, provide more accurate reporting of demographics formarket research and applications thereof.

Examples disclosed herein solve problems arising from uses of computertechnologies. Namely, automated information gathering systems (e.g.,computer data gathering systems) do not differentiate between true andfalse user information. For example, a computing system accepts userinput data, such as data received via a registration process asdescribed herein, as truth. Where such data is not truthful (e.g.,wherein a subscriber to a database proprietor mistakenly orintentionally provides false information), the untrue data isunknowingly processed by the computing system as truth. Such untrue dataprocessing can be compounded when the untrue data is manipulated and/oranalyzed to create conclusions and/or new data (e.g., false demographicsassociated with an impression lead to false conclusions and incorrectand/or unreliable predictions or audiences exposed to media based on thedemographic impression data). For example, if a 12-year old malesubscribes to a database proprietor (e.g., Facebook®) as an 18-year old,then an impression for a media presentation of SpongeBob SquarePants maybe incorrectly associated with an 18-year old. Parties receiving thisdata may incorrectly assume 18-year old males are interested inSpongeBob SquarePants and may develop marketing campaigns directlytargeting this demographic. These marketing campaigns would likely nothave the anticipated impact because they are based on defective dataunbeknownst to the computer data gathering system.

Examples disclosed herein are useful to compensate and/or correct forreception of defective data by providing countermeasures, whicheffectively convert instances of untrue user input into accuratetruthful representations. These accurate truthful representations areassociated with household impressions. Further, household members areassigned viewership of the impressions. The example methods andapparatus build models using the above-disclosed corrections andassignments of viewership as a training set. Thereafter, the models areapplied to database proprietor demographic databases to correct theuntrue demographic data and correctly associate the demographics withimpressions.

Although certain example methods, apparatus and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus and articles of manufacture fairly falling within the scope ofthe claims of this patent.

What is claimed is:
 1. A system comprising: at least one memory;programmable circuitry; and instructions to cause the programmablecircuitry to: obtain first time information corresponding to one or morefirst times that a panelist accessed first media via a television;obtain second time information corresponding to one or more second timesthat a household accessed second media via an over-the-top device, thesecond time information matching the first time information andassociated with corrected demographic information of members of thehousehold; determine that a first genre of the first media matches asecond genre of the second media; determine that a member of thehousehold accessed the second media via the over-the-top device based onhistorical media access events of the panelist; and attribute the accessof the second media to the member of the household to create a correcteddemographic impression record.
 2. The system of claim 1, wherein theprogrammable circuitry is to utilize the corrected demographicinformation to access at least one demographic bucket, the member of thehousehold and the panelist associated with the at least one demographicbucket.
 3. The system of claim 2, wherein the programmable circuitry isto: obtain third time information corresponding to a total time thefirst media was previously accessed by a group of panelistscorresponding to the at least one demographic bucket, the group ofpanelists including the panelist; and determine a probability that themember of the household accessed the second media based on dividing thefirst time information by the third time information, wherein the firsttime information matches the second time information and the probabilityis indicative that the member of the household likely accessed thesecond media via the over-the-top device.
 4. The system of claim 1,wherein the programmable circuitry is to determine whether to attributethe access of the second media to more than one member of the householdbased on the historical media access events of the panelist.
 5. Thesystem of claim 4, wherein the member is a first member and theprogrammable circuitry is to: attribute the access of the second mediato all members in the household; and execute an elimination analysis todetermine whether to eliminate a second member of the household from theattribution of the access of the second media.
 6. The system of claim 1,wherein the member is a first member, the panelist is a first panelist,and the historical media access events of the panelist are firsthistorical media access events of the first panelist, the programmablecircuitry is to: attribute the access of the second media to all membersof the household; compare the second media to second historical mediaaccess events of a second panelist, the second panelist havingdemographics corresponding to a second member of the household; andremove the attribution of the access of the second media from the secondmember of the household when the second media conflicts with the secondhistorical media access events of the second panelist.
 7. The system ofclaim 6, wherein the programmable circuitry is to remove the attributionof the access of the second media from the second member of thehousehold when the second time information conflicts with third timeinformation corresponding to one or more third times that the secondmember of the household historically accesses media.
 8. A non-transitorymachine readable storage medium comprising instructions to causeprogrammable circuitry to at least: obtain first time informationcorresponding to one or more first times that a panelist accessed firstmedia via a television; obtain second time information corresponding toone or more second times that a household accessed second media via anover-the-top device, the second time information matching the first timeinformation and associated with corrected demographic information ofmembers of the household; determine that a first genre of the firstmedia matches a second genre of the second media; determine that amember of the household accessed the second media via the over-the-topdevice based on historical media access events of the panelist; andattribute the access of the second media to the member of the householdto create a corrected demographic impression record.
 9. The at least onenon-transitory computer readable storage medium of claim 8, wherein theinstructions are to cause the programmable circuitry to utilize thecorrected demographic information to access at least one demographicbucket, the member of the household and the panelist associated with theat least one demographic bucket.
 10. The at least one non-transitorycomputer readable storage medium of claim 9, wherein the instructionsare to cause the programmable circuitry to: obtain third timeinformation corresponding to a total time the first media was previouslyaccessed by a group of panelists corresponding to the at least onedemographic bucket, the group of panelists including the panelist; anddetermine a probability that the member of the household accessed thesecond media based on dividing the first time information by the thirdtime information, wherein the first time information matches the secondtime information and the probability is indicative that the member ofthe household likely accessed the second media via the over-the-topdevice.
 11. The at least one non-transitory computer readable storagemedium of claim 8, wherein the instructions are to cause theprogrammable circuitry to determine whether to attribute the access ofthe second media to more than one member of the household based on thehistorical media access events of the panelist.
 12. The at least onenon-transitory computer readable storage medium of claim 11, wherein themember is a first member and the instructions are to cause theprogrammable circuitry to: attribute the access of the second media toall members in the household; and execute an elimination analysis todetermine whether to eliminate a second member of the household from theattribution of the access of the second media.
 13. The at least onenon-transitory computer readable storage medium of claim 8, wherein themember is a first member, the panelist is a first panelist, and thehistorical media access events of the panelist are first historicalmedia access events of the first panelist, the instructions to cause theprogrammable circuitry to: attribute the access of the second media toall members of the household; compare the second media to secondhistorical media access events of a second panelist, the second panelisthaving demographics corresponding to a second member of the household;and remove the attribution of the access of the second media from thesecond member of the household when the second media conflicts with thesecond historical media access events of the second panelist.
 14. The atleast one non-transitory computer readable storage medium of claim 13,wherein the instructions are to cause the programmable circuitry toremove the attribution of the access of the second media from the secondmember of the household when the second time information conflicts withthird time information corresponding to one or more third times that thesecond member of the household historically accesses media.
 15. A methodcomprising: accessing, by executing an instruction with programmablecircuitry, first time information corresponding to one or more firsttimes that a panelist accessed first media via a television; accessing,by executing an instruction with the programmable circuitry, second timeinformation corresponding to one or more second times that a householdaccessed second media via an over-the-top device, the second timeinformation matching the first time information and associated withcorrected demographic information of members of the household;determining, by executing an instruction with the programmablecircuitry, that a first genre of the first media matches a second genreof the second media; determining, by executing an instruction with theprogrammable circuitry, that a member of the household accessed thesecond media via the over-the-top device based on historical mediaaccess events of the panelist; and assigning, by executing aninstruction with the programmable circuitry, the second media to themember of the household to create a corrected demographic impressionrecord.
 16. The method of claim 15, further including using thecorrected demographic information to access at least one demographicbucket, the member of the household and the panelist associated with theat least one demographic bucket.
 17. The method of claim 16, furtherincluding: accessing third time information corresponding to a totaltime the first media was previously accessed by a group of panelistscorresponding to the at least one demographic bucket, the group ofpanelists including the panelist; and determining a probability that themember of the household accessed the second media based on dividing thefirst time information by the third time information, wherein the firsttime information matches the second time information and the probabilityis indicative that the member of the household likely accessed thesecond media via the over-the-top device.
 18. The method of claim 15,further including determining whether to assign the second media to morethan one member of the household based on the historical media accessevents of the panelist.
 19. The method of claim 18, wherein the memberis a first member and further including: assigning the second media toall members in the household; and executing an elimination analysis todetermine whether to eliminate a second member of the household from theassignment of the second media.
 20. The method of claim 15, wherein themember is a first member, the panelist is a first panelist, and thehistorical media access events of the panelist are first historicalmedia access events of the first panelist, and further including:assigning the second media to all members of the household; comparingthe second media to second historical media access events of a secondpanelist, the second panelist having demographics corresponding to asecond member of the household; and removing the assignment of thesecond media from the second member of the household when the secondmedia conflicts with the second historical media access events of thesecond panelist.